What it is

Plakwerk is a signmaking event that was held at our school, and we made an interactive projection for the entrance. When you walked in and looked at the wall on your left, a little robot suddenly appeared out of nowhere and started following you. If you walked on, it came with you, and if you walked back, it went back too. We only gave a robot to a few people, because one for everyone would have made the wall way too busy.

Across from it we added a smaller projection with another robot. That one was just a video, but every now and then it would say things like "Hey, welcome to Plakwerk" or ask if you'd paid for parking.

How we made it

I made this with one other person, so it was just the two of us. We built it in TouchDesigner and spent a lot of time on it. The first step was trying out tracking software to see if it would even work, and that was a real struggle, because we were both still new to TouchDesigner. We modelled and animated the robots in Blender.

At some point we bought a better camera, a depth camera. It measures distance, so it can really see people's bodies, and that made a big difference. In TouchDesigner we took a visitor's position, smoothed it out and used it to move the robot. We also wrote some Python code that decides when a robot flies, waits or waves, and we did some webcam tracking too.

How it went

People really liked it. At one point it crashed, so we had to work really fast and put together a lighter version. Daylight was a problem too, because it made the projection hard to see. It really works best in a dark room, which was a bit of a shame.

The tech behind it was pretty complicated and it took us a long time. But it was all still pretty new to us, so I think it turned out really well. People also thought it was really cool that just the two of us had made it.