How to use frame averaging as a simple method to smooth keypoint data from PoseNet in p5.js. — Google’s Tensorflow.js PoseNet model is extremely useful. It returns real-time pose estimation data in the browser from just a webcam, which allows for all sorts of accessible embodied interactions (like novel navigation modes and online dance parties) in the browser. Using the ml5.js …

Documentation and reflection on “Recreating The Past,” a ten-week course on computational art taught by zach lieberman at the School for Poetic Computation. I had the opportunity to take Zach Lieberman’s Recreating The Past course this summer. …

Machine learning is paving the way for the internet to be a source of creativity and collaboration — This is the second of two articles about designing for machine learning on the web. This article addresses machine intelligence as a creative collaborator; the first discusses the body as input. This series of articles was originally published by the Machine Intelligence Design team at Adobe Design.

With machine learning, our bodies may be the next step in the way we interact with the internet — This is the first of two articles about designing for machine learning on the web. This article discusses the body as input; the second addresses machine intelligence as a creative collaborator. This series of articles was originally published by the Machine Intelligence Design team at Adobe Design. I recently attended…

We’ve given computers our minds. We now need to lend them our bodies. — This article was originally published in Adjacent Issue 4: Bodies and Borders Computers are ubiquitous. Aided by the internet, they have infiltrated our individual, social, and political lives. With the expansion of artificial intelligence and machine learning, they are poised to have even greater impact. …

Learnings and workflow from sending 3D color and depth images across networks in real time. — A few years ago I started working on an open source tool called Kinectron, affectionally taking its name from its two major components: Kinect + Electron. …

Avoiding common pitfalls when working with cameras and other sensors in public settings. Back in 2016, I built Kinectron, a tool that sends volumetric and skeletal data online using the Microsoft Kinect. Since then, Kinectron has become widely used by students at NYU ITP, especially in the end of semester…

One of the most frequent questions I get from people working with the Kinect, and Kinectron is: what is the skeleton actually made up of, and how is that represented? …

We visualized the body electric of three contemporary circus performers on a 120’ x 10’ screen in real time. These are the data considerations that most shaped the project. — When Aaron Parsekian and I found out we would have the opportunity to program the IAC media wall, we knew immediately we would create an experience working with data and live performance. We had seen a lot of performances using motion capture — the performer moves and the visualizations respond…