CCN San Francisco: my first international conference in 2.5 years. After a hectic summer (and year!), getting covid for the first time a few weeks before the conference, I was tempted to skip it. How glad I am that I didn't!
The conference exhibited an impressive range of topics, but somehow still retained a sense of coherence. From experimental work in perception to decision making, with data types ranging from behavior to neuroimaging and neurophysiology in rodents to humans, many of which were accompanied with some modelling approach. These also ranged from cognitive models to varieties of deep and recurrent neural networks. It didn't feel like there was a bias toward or away from any model organism (species) or computational modelling approach - as long as you’re working in the cognitive or computational mind or brain sciences, you're welcome here!
It felt cutting edge, relevant, and at the very least, full of interesting work for cognitive neuroscientists with a computational bent or ML folks with a cognitive/neuroscience bent. As cognitive or neuroscience conferences often have a strong empirical bias, and computational cognitive or neuroscience conferences sometimes focus more on certain types of models or overemphasize animal work, CCN fills an important gap in the cognition-computational-neuroscience conference scene.
Brief personal highlights
Some highlights of mine which also illustrates the breadth of the conference:
In Chelsea Finn's talk, I learnt that getting actual physical robots (with a deep neural network; I believe deep reinforcement learning) to learn tasks and generalizing in real life is *much harder* than in a simulated environment (e.g., Go, ATARI games). Tasks included training to pick up an object and dropping it in a designated area. Testing on the same object doesn't give 100% accuracy, and when robots were tested on a novel, different color/shaped objects and environment, they failed a lot! I think overall performance was 10-30% if I remember right (lower than you'd expect anyway). Very interesting to know how far we are from human-level AI in the real world.
Talks on "Drivers of learning across timescales: evolutionary, developmental, & computational perspectives" - the evolutionary perspective was different to our standard cognitive / computational / neuroscience talks. Interesting and intriguing.
I joined the tutorial "Varieties of Human-Like AI" a very good tutorial with code on the basics of RL from Q-learning to successor representation to model-based RL in simulated environments like grid worlds. A bit fast but not unusual. Some nice suggestions were to have longer tutorials or hackathon-style days where people could hang out to chat or code whenever they wanted to.
Innovations
I have only been to CCN once – the first one in New York City in 2017. It was very nice, but it was mostly a standard conference. Over the last few years, I heard about the new innovations at CCN (and Neuromatch), but have not been able to attend. Joining the conference with some skepticism, I can say that I felt the 'innovative' bits were generally good and purposeful. I also think it took a dedicated group to make it so – if it were just a large old conference trying to do a few new things, it would probably end up gimmicky. As I am typically a passive participant at conferences, and only participate if I have to, I was skeptical of these ideas. But it ended up being rather fun.
Mind matching - their algorithm (by Neuromatch, and of course they’re now using a large language deep neural network model!) matches people with similar interests. We provided several abstracts and the model processes the text and matches us with other attendees. 3-4 people join a table, get ~25 minutes to chat, then go onto the next table. 3 tables each. From what I heard, sometimes it was a bit random, other times “too good" (with people you knew – there’s an option to not match with specific people; though you have to know in advance they’re here…), but all in all it worked pretty well. I met a few interesting people, and there were 3 (out of 11) who I might not have met otherwise, and we chatted in the conference and over drinks (there were others but I would've met them through friends).
General Adversarial Collaborations - I won't say too much about these as there's an explanation of them here: https://gac.ccneuro.org/call-for-proposals. What I did find interesting was in one, there was time to discuss the questions posed as a small group (everyone was assigned a group in the audience). As the questions were stimulating and interesting, our group had some fruitful conversations about the definitions of mental simulation: does it have to be sequential, we probably do simulate but it depends on time/energy constraints, and we do simulate but can be very bad at it! It would've been even better if there was more debate on stage, but no one can control this (and quite a few zoom panel participants made it more difficult).
Reviewing submissions - this was an interesting process 2-page submissions with a scoring system to select talks. Reviewers had to comment on the submissions (I had 7) on their potential impact and clarity, and produce ratings, which were normalized and aggregated for talk selection. Most interestingly for me, we got feedback from a range of reviewers who ranged from out-of-your-field to expert (within the CCN community). Normally we only get reviews for experts of some sort on your topic. It was interesting to hear comments from others in the more ‘general’ computational fields, and what aspects of the work were unclear or interesting to the non-experts. This is feedback we don't normally get.
I suppose one criticism of using this system to select talks is that the most highly rated ones are likely submissions where the authors could convince researchers from a large range of computational approaches both in and out of their field that it's important. This probably leads to a bias away from interesting approaches that are a bit more niche or different or dare I say, ground breaking, in comparison to more established and well-known approaches, and certainly labs. To be fair, this is also often true with other reviewing systems. Said that, it's a very cool system, and what's innovative ideas for if not to keep having and improving them? Huge credit to CCN and the team who did this (see https://meadows-research.com).
A half-baked idea: [walking tour-inspired] multiple poster presentation starting times
With all the innovation, I couldn't help but notice one aspect that was traditional and suffered from old problems - the posters! I am sure others have felt this before - presenter presents to one person, then others join halfway. Sometimes people can get the gist and follow, but often they can't. Some decide to come back later, but often come back to find the presentation is mid-way again! Plus, as the presenter, you often want to see other posters in your session, as these are probably the most relevant ones!
The idea is inspired by how walking tours works – with multiple tour starting times to guide each group. People would arrive just before the start time so it's possible to wait for a few people to gather before the presenter starts. Note that this is not the same as a time slot for when the presenter is around, but the presentation *start* times. Of course, it should be flexible, e.g., it might take longer than you think. Ideally, there’s a conference-wide system where start times are staggered across posters (maybe randomization is fine, with future plans for an algorithm!). I acknowledge that sometimes the 1-on-1s are great, but this can also happen, such as when it's not a designated “tour” in between start times, or after the session.
This would make it more likely for the presenter to present to multiple people at once, rather than give multiple 1-on-1 presentations, which can be exhausting. It also lets the presenter take a break and see other relevant posters in their session (which everyone wants!).
Issues
Some minor issues were raised which many people agreed with. For example, there needed to be more social time, maybe a social night - this can be organized by a group of students and maybe postdocs, who would know what's fun. Maybe lunch each day could be provided so people could stick around, or at least organized somewhere.
There were probably too many zoom speakers and discussion panelists. Maybe it's a post/peri-covid phenomenon, and hopefully it'll get better. We should be flexible for speakers, and it makes sense when people really can't come (e.g., family, unexpected issues) but it felt like there might’ve been a few too many this time.
Future of CCN
Finally, I look forward to the future of the conference. It started with great promise, and has since been growing and maintained by a group of passionate and dedicated young leaders. As many have heard, CCN got into a bit of financial trouble. As a young conference, having covid suddenly stop everything meant they apparently suffered a bit financially from cancelling the 2020 conference. What's clear is that the organizers are now doing their thing, planning to make the next CCN work and many more in the future.
My (probably naïve) ideas on keeping the conference going with healthy finances in the near future (inspired by others at the conference): there could be a limited number of places for where the conference is based (for now at least). For example, a particular venue or hotel (get deal over a few years) or university with a big center. Another is pairing with other conferences – people said that pairing with Bernstein at Berlin worked well, boosting interest for both events.
All in all, it was a great conference, and I truly hope it will continue for many years to come. Major thanks to the organisers and the sponsors! Hopefully current and potential sponsors can see this for what it is – a truly special conference with a bright future. To paraphrase the saying: if you build it and keep it going, we will come!
(Picture credit to Laurence Hunt at his great closing speech)
No comments:
Post a Comment