Vector Robotics 2025 Retreat

So the weekend begins to a pretty rocky start - none of the new hardware is working, none of the new embedded software is tested, we're both slightly sleep deprived after a long night trying to get the hardware assembled and working, and to top it all off, I was starting to feel pretty ill.

Despite spirits being slightly lower than we had hoped for, we were both pretty excited for the weekend , and by midday we were piling boxes of electronic components, robotics parts, test equipment, tools, rework kit, and (after a quick pit stop) plenty of food into our unsuspecting studio airbnb.

It was pretty clear we weren't going to be hitting most of the goals we'd planned for the weekend though, so what did we get up to?

Day 1 - Saturday

My notes from the first day for the blog were as follows:

• Henry fixed the FPGA
• Henry fixed the motor driver PSU
• Henry fixed the CAN Pi Helmet
• Henry drove us there
• Burger bite rolls are cracking
• let's go!!!

A little more detail on those - first off, Henry made some sausage rolls but with burger meat instead and some flaky salt on top for snacks on the drive down - there's incredible and I don't know why they aren't a thing. Give them a try sometime!

Anyway, onto the issues that Henry sorted:

I'll start with the Pi Helmet, lot's of strangeness going on here including a very interesting problem where it would only (half) boot after you put a drop of IPA on it. After about 10 different attempts at reflowing pins on the buck controller IC (at which point we were on the third chip). Effectively this all boiled down to an assembly issue although there were plenty of red herrings along the way with the internal VCC not being generated sometimes, occasionally generating close to 3.3V (although we think this was just random chance), and sometimes enabling the power-good LED (not sure what's going on there - this was only on the second one so maybe it got a bit toasty during rework?). Anyway, after the final reflow we had to put back about 5 or so parts that we'd removed while trying to debug this issue. Plenty more functionality to test here but good news that it's now generating 5V for the Pi, and only four hours lost...

Next the motor driver board - similarly some reflowing of joints on the STM & motor driver IC managed to resurrect the power and allowed programming of the FPGA (luckily not nearly as much effort required here as on the CAN helmet board and no chip swaps wither). Didn't have enough time to test much of anything else on this board but it put us in a good position for progress the next day.

There's a lot of Henry here, so what was I up to during all of this? Well I helped a bit with some of the debugging, and did some software development preparing for when things might be working, but was mostly trying to regain my energy after being ill. Good thing Henry was able to pick up the slack on that one (I'm very happy that we both have a decent amount of multidisciplinary skill and can do this for most bits of the project).

We're so back!

Day 2 - Sunday

It's so over.

I don't have very good notes from this point on as I started to lose my mind with some of the issues, but I've reconstructed the events from memory as best I can. Proper testing the motor driver could proceed now that the STM and FPGA were programmed & it turned on by itself - this revealed some issues.

The simpler of the two was that the settings system wasn't working (although beyond that, it wouldn't even read from the EEPROM anymore). A quick visual inspection revealed an issue with the I2C clock line solder joint on the STM so we fixed that. And now the STM wouldn't boot - it didn't take long to figure this out, but probably longer than it should've - this had connected a pullup to the BOOT0 pin which was now putting it into a bootloader mode rather than booting normally. After installing the program to allow editing of the option bytes and telling it to ignore the BOOT0 pin, (and a few minor bug fixes and improvements) settings were working well!

The next issue is that the position values from the motor encoder were coming back as the same value every time (I think zero, but I can't remember fully). After a really long investigation we didn't feel any closer to finding the solution. I assembled another one, since it's not too taxing, to see if it was an issue with the chip. It did the same things, so they're consistent at least.

While I was losing my mind with the motor driver encoder, Henry was busy doing some of the software integration on the Pi Helmet. He actually had some good luck on that and he was able to connect to the CAN controller, enable loopback mode, and receive the messages he was sending. Great stuff there!

We'd been hoping to spin a motor today at the very least, but run into a handful of issues again (all still assembly rated so far).

Day 3 - Monday

We're so back again! Henry came to the rescue again and figured out the encoder issue before I was even out of bed - when changing the interface between the driver and encoder boards I'd moved the driving source for the MOSI data line onto the encoder board - this is fine since reading the position just requires writing all 1's, however I'd accidentally pulled it low. So just one hardware/design issue on the new set of boards so far (not too bad in my eyes). After a quick swap to pull MOSI to 3V3 we started getting proper data out of the encoder and motor driving could begin.

Plenty of miscellaneous bugs in the code for this and so most of my day ended up being spent of improving the debug helper functions which would allow me to see what it's doing and control the correct parts of the logic in the right ways. Frustrating that it took so long to do this but it needed to happen at some point so it's not like it's wasted time (story of the whole retreat to be honest).

The encoder wires decided to snap again which I didn't realise for a while and let me down a rabbit hole of debugging that wasn't necessary, but after fixing that and at the end of plenty more messing around, I was finally able to spin a motor. Only I had far less control over it than I'd anticipated. It's so over again...

Alright it's not that bad, but while what you see above might look good at a first glance, probably not so much if I say that it's being commanded to drive with zero torque. With the power of hindsight, having now fixed most of them, there were a fair amount of issues that were never going to be fixed by the end of the retreat. At the time though it was fairly clear what the rough nature of the issue(s) were - there was a runaway feedback loop (with positive feedback rather than negative). It would have gone far crazier except for a handful of software limits that I'd set in a rare act of forward thinking. This was as far as the motor control on the new hardware made it (about where it was before...) although I have since got it working kinda ok. There's a lot more tuning (and more refactoring) to do before I show off those results though.

Henry had made some good progress in the meantime with the software on the Pi and had gotten a number of various parts working (to the demo/proof-of-concept stage at least). In no particular order these were:

• CAN controller (MCP2517FD) mostly from the day before
• 9DOF IMU module (BNO55)
• Addressable LED strip output (WS2812B)
• 2D 8x8 LiDAR module (VL53L5CX)
• 2D circular LiDAR UART control (D200/LD14P)
• Running the Pi from an NVME drive

Sometimes I wish I was working on the higher level software, but I think I'd probably just want to be working on embedded things if I was. Bit of a grass is always greener situation...

Summary

Tuesday was spent packing up, driving home, and doing literally anything else that wasn't robotics.

Like I said before, the progress made was absolutely way below what we'd hoped, but most of everything we did needed to happen and so the retreat probably cut short about 2 or 3 months of work that we only realised was there in hindsight. We also had a great time so I'd call it a huge success and we're both pumped for the Vector Robotics 2026 Retreat!

I'd like to take this moment to recommend this - doesn't have to be for robotics, but just taking some holiday to work on some cool projects or hobbies with friends is great fun. Even better if you can go away somewhere for it rather than just staying at home - I find a hot tub has a way of washing away your troubles and revitalising you if you're working on anything remotely frustrating.

The blog is getting close to being caught up with the current state of progress (at least on the electronics & embedded software side of things), I have a couple ideas for a post or two, but I imagine the rate of posts will go down again after the end of Jan. Maybe Henry can take up the slack for a bit and post about the past year or so worth of mechanical & high-level/application SW? wink wink nudge nudge

Comments/Notes From Henry

As far as I can remember, Max's rendition of the retreat feels pretty accurate. I felt like an engineering god on Day 1, but in hindsight I think that was mainly because Max was battling illness and skill issues. Our goals were certainly ambitious, but thats never a bad thing in my mind... If anything, it's a very good excuse to eat more snacks and drink more beers in order to pretend it's all going great!

I spent a lot of time re-familiarising myself with the high-level code that I had written multiple months before, the code base is getting pretty complicated now (probably due to my above point about having out of reach goals...). Good progress was made here though, with demo examples for all of our planned sensors available now. In hindsight, I also sunk a silly amount of time into optimising the run-speed of the python control and simulator (Yes a custom simulator!!) loops, hopefully that work will pay off in the long run, it can run WAY faster than it needs to currently.

It was nice to get stuck into some hardware again on Day 1. Due to Max's slow progress on the motor controller, it was pretty clear we weren't going to be able to test the latest revision of the BeltBox (yeah, its finished and there is no blog post about any of it - I know...) and I didn't have any non-software tasks. Getting stuck back into some SMD rework was also fun, I haven't done that in a good few years and it was good to see I'm still amazing at it.

Max's comment about recommending this sort of "hackathon" is very true, my writing isn't quite as motivational as his, so just go read that paragraph again. Oh and remind me to take more photos next time, I took one photo and it wasn't even of any of the progress we made!

On an absolutely wild off-topic, but very noteworthy, thing that happened: One of Vector-Robotics favorite songs is "King of the Slugs" by FatDog. To cut a long story short, a slug fell off of the ceiling into my bed early one morning, waking me up and leaving me VERY confused. I considered if Max had had some sort of wild fever dream and put a slug in my bed before working out that it had fallen from the skies above, but that seemed a bit unhinged even for him. I am now the self-proclaimed King of the slugs, and I decree that you should all go and listen to the song. Also, sorry to whoever's spotify account was left logged into the Airbnb TV, FatDog WILL make an appearance in your spotify wrapped next year.

I'm starting to get the feeling that Max would really like me to pull my weight with the blog posts for some reason. This is actually something I want to do, I just need to stop making actual progress on the robot and set aside some blog time. I mean, we do all realise that Max only writes so many post so he can avoid integration hell right??? I promise there will be some Mechanical development posts coming soon, and some high level software posts just over the horizon. So buckle your seatbelts and assume the brace position for some of the best posts this blog has ever seen.

With alarming confidence,

Henry