Albatross

Back in 2015, I saw Idina Menzel at a concert where the audience was encouraged to take photographs and tweet to their heart's content. A lot of fun things happened on stage that night and it occurred to me that there was no easy way to obtain a complete collection of everything posted by the participants.

So I wrote Albatross, in part to scratch an itch, and in part to learn more about the asynchronous features of Python. It does some really crazy stuff, dumping every tweet it finds into a queue server and then consuming them into a live-compressed archive and generating statistics from the process.

The front-end stuff was a lot of fun too: maps showing tweet location, ring charts with tweet statistics, a javascript-based word cloud, and an image viewer that changes the size of the images based on the proportional popularity of the tweet in question.

The original plan was to build it out as a SAS site, called "TweetPile". The idea being that anyone could trigger a collection and generate stats from them, but I soon realised that the stream Twitter allowed you to access wasn't actually all the tweets, and then they started mucking with the order of tweets in general and it all sort of came apart. Eventually Musk bought Twitter and set fire to it, so I guess this was my own lesson around building a product on top of corporately-controlled system.