Refresher on Force Directed Graph

Force Directed Graph is a free and open source JavaScript library. It allows you to feed in a data set of “nodes”, essentially an arbitrary object with properties like color, size, position. You then also feed in “links” which are references to which node should be connected to another node with a line. Documented node properties are packed into formats that can be read on a device’s graphics card. Then a physics simulation that disperses the nodes and links on every frame while the application is running. Lastly, the nodes and links are drawn to the screen in an efficient way. There are many graph libraries that display and run this type of layout. However, this library is unique because the physics simulation and the rendering are primarily running on the graphics card. This allows for many calculations to happen in parallel. For example, calculating the position, velocity, and forces between thousands of nodes sixty times a second. The results are that the nodes can be in 2D or 3D space and the drawing of thousands of nodes and links can animate.

Animating Related Artists

This Force Directed Graph is really good at animating thousands of nodes efficiently. This feature is something that I felt was lacking in previous versions of my Shape of Art studies. I had spent a lot of time creating the dataset. I was proud that I had a list of about 100,000 artists and a way to link them. Plugging the data in, the graphs move. So, it was impressive that there was a lot of nodes on the screen. But, it was not materially different from what other libraries offered. I became focused on figuring out how to make the animation more dynamic. Partly because of the library I built. Partly due to the texts of Kandinsky that inspire this inquiry in the first place. His writing has a reverence for artists, the dissonance they create for their generation, and how future generations use that as a reference point for style. When I read his text, I get the sense that there is a push-and-pull between artists that are aware of each other. This latest feature attempts to address this by revealing nodes over time. At the bottom of the latest prototype, you can drag a slider left-and-right to reveal new artists. The artists date back all the way to 970. This gives historical context, but it also heightens the tug-of-way when a new node or artist is revealed in the graph.

Labeling Artists like a Point of Interest on a Map

The second aspect of previous studies I made was that visitors had to hover their mouse over a node in order to see the name of the artist that node represented. I imagine Edward Tufte would describe this as a poor visualization. In The Visual Display of Quantitative Information he said, “Above all else, show the data”.3 My studies had lots of nodes, but not a lot of data. You could tell the nodes are connected, but to who? And why? So, I went down a path to add labels to all nodes. The first attempt I made, it was all black. There are so many nodes that adding labels that were in a font size large enough to read made visual cacophony. One night driving I was looking for directions and zoomed out of my map. Eureka! This was how I should display the labels. In maps apps, labels are only rendered at specific zoom levels. You do not need the name of all streets if you can see an entire state. The logic that I imparted to Force Directed Graph was a priority system. It prioritizes labeling nodes that are in view. In the 3D environment we render to, this means a few things. First, there is a near and a far clip space. Basically, the camera does not need to worry about seeing anything that is too close or too far. Second, it prioritizes labeling nodes who have a lot of connections. Third, it discourages labeling nodes whose neighbors are already present. Fourth, the developer can always override this in a couple of ways: either force the priority on a specific node or alter a library level property called “obscurity”. The obscurity basically loosens or tightens these priorities as a zero-to-one slider. At an obscurity level of zero, you can see all the labels. The cacophony I started with. At an obscurity of one, you cannot see any labels. And at an obscurity of 0.5, 50% of labels can be visible. This four tiered system at an obscurity level of 0.9 is what you see in the current prototype (recorded in the video above). It is fairly simple compared to a Map application, but its behavior is reminiscent of the inspiration.

Try the latest prototype

These two features have been months in the making. You can tell because the visual aesthetic of this prototype is pretty barebones. Personally, this version speaks to me more than the previous ones. It has more data. It is more alive. While not visible, I have also baked in color coding based on output formats defined in Artsy’s Art Genome Project.4 I am excited to add color to the next version. If you give this latest version a try, I would love to hear your thoughts.

–Jono

What goes into a museum membership?

Hopefully when you look at the tiers above, it is easy to discern the tiers. It is a sliding scale in price and in benefits. But, how did we come to this specifically? During my time at NEW INC, I learned a bit from meeting museum operators.4 Working on Dataland opened up my eyes further. My colleagues at Constellation Immersive started by putting together a survey of what types of benefits currently exist.5 This ranged from membership offerings of established museum to more avant-garde ticketed experiences. Numerous meetings over the years were held to discuss, at a high level or put another way: from the consumer’s perspective, what “felt right”. A natural compliment emerged of different motivations: maximize memberships and offer a truly unique experience.

To maximize memberships may sound terse, but the reality for a new, and particularly small, institution like Dataland is that memberships are vital. Operationally, memberships account for a majority of stable revenue. Beyond monetary incentives, having a strong membership group establishes a clear value proposition. And a museum’s members can guide what that value proposition ultimately becomes. How it operates and what should be on display. The museum serves the community and a clear membership group clarifies this.

To offer a truly unique experience starts with the exhibition, right? It can. But, it does not always. The presentation of tickets and memberships convey expectations for a visitor. By presenting benefits, a museum frames what kind of experience lies ahead. Everything from the price, through the member benefits, to the names frame the experience. Is this a high brow or casual experience? Is there one gallery or many and how many exhibitions are there? The membership can begin to communicate this. From the meetings I was in, it was clear that the founders appreciate an agency from their audience. They encourage participation and imagine that memberships with a bit of whimsy can foster this.

Balancing these two competing motivations brought us to: Explorer, Dreamer, and Pioneer memberships. Practically speaking, Explorer is your typical individual annual museum membership. Dreamer has perks that make it a good value for a household. Lastly, Pioneer is a uniquely exclusive tier for collectors, offering both access and art objects. That is the vision behind the memberships. To fill out each tiers’ offerings, however, required more back-and-forth. This time that dialog occurred with our ticketing vendor. The team had to come together to balance how we could make good on each benefit. The discounts and Member Access, for instance, are conveyed and managed by the ticketing vendor. We worked with them to define what was possible. For other items, like Priority Gallery Entry, we defined commonsense solutions like marking tickets different colors for employees to easily verify a visitor’s priority. Each benefit requires an accord among the systems we employ and operate. We had the vision and over time we chiseled away at potential benefits. The chart above is the result.

Learn more about Dataland Memberships

It took almost the entire three and a half years to come up with the tiers and benefits. The implementation, on the other hand, took a couple of months. While the chart may look straightforward, the systems in place to convey the chart are more complex. We rely on numerous vendors in order to achieve the membership experience. A lot of my job in the context of memberships, has been to make sure that these systems are sharing just enough. What does “just enough” mean? It comes from Principle of Least Privilege or PoLP.6 In cybersecurity circles, PoLP is a methodology of sharing that reduces a potential attack surfaces. This is done by granting a service the minimum permissions necessary to perform a job. In the case of Dataland, we give the minimum information necessary for a vendor to perform its job. It is in part for security reasons, but it is also in part for modularity. If we can reduce the inputs necessary for a module to work, then we can change that module down the road with minimal impact to the system as a whole. My hope is that this “just enough” method serves the museum well as it matures, like supporting languages other than English.

So, that is a look on how we landed on memberships for Dataland. I hope it has piqued your interest in Dataland or its memberships. It certainly has been a challenging puzzle to work on over the years. As always, feedback is greatly appreciated.

—Jono

Officially titled the Creative Growth Strategy Sprint, it was six 90 minute sessions filled with digestible activities to cultivate your business as a lifestyle.2 It started the first week of January and was held on Wednesdays. The cohort consisted of solo business owners and freelancers from Europe, the UK, and the east coast of the US. Matilda laid out terms like Type 2 Growth.3 An alternative growth structure to the up-and-to-the-right hockey stick of positivism. She also gave us building blocks to make a Growth Donut (pictured above), a way to quantitatively look at our time and put boundaries around what activities we engage with. This was used to great effect to reveal the activities that I did not like or want to do. Each 90 minute session included: some personal time for reflection and jotting ideas down, small breakout groups to talk freely and get to know your cohort, listening time to hear about the theme or concept of the week from Matilda, and reflection time to ask questions and recap what we went over. As Matilda is based in London and I am in Los Angeles, these sessions were first thing in the morning for me. It was a great albeit sometimes challenging way to start a hump day. Despite the time difference, I always felt energized by the time the session was over at 7:30 in the morning with the sun barely even rising.

I came in holding a couple of thoughts from the buzz of the New Year. First, Dataland is opening this spring.4 As excited as I am about the launch of the museum’s in-person experience, as a business owner I see this as a crucial moment to evolve my business. The launch of Dataland presents case studies in how I would like to work with future clients. I want to capture that to make the next five years of my business as full of growth as the first five have been. Second, I wanted to create a better pipeline to produce social media content. Ever since the pandemic, I have withdrawn from trying to grab the potential five second spotlight of social media virality. Recently, I miss structuring my own visual work in a way that I can share it online. I wanted to use Matlida’s sprint as a way to refocus and make good on these motivations. The sessions were eye-opening. As each week passed, it was more obvious that social media as a format for producing content just did not match with the type of experiences I want to produce; nor does social media offer the value I truly seek for my business. The sessions also gave me better insight into how I should go about putting together case studies for landing that next big client through smaller actionable steps: create client personas, send out feedback questionnaires to current and former clients, see the landscape of who my competitors are. Do these smaller steps represent opportunities to collaborate with others? If so, that could be the spark that starts a bigger relationship.

See my Growth Strategy document

In all, the six sessions built on top of each other. They also presented different analogies and games or lenses to understand what you are doing. Each week we filled out a notion template that Matilda created for the sprint. At the end of the six sessions we had a robust Growth Strategy document (click the button above to see mine). It has the Growth Donut, brand taglines, keyword diagrams of what makes work important to me, a competitive landscape of other inspiring creatives in my space, and activities that I can do to make meaningful progress toward my longterm goals. Or as Matilda describes when landscaping our garden, our Big Ol’ Tree. The document is my personal output and accountability to stay focused and channel all that good intention from the beginning of the year. But, I will remember the Creative Growth Strategy sprint for the journey and colleagues I met along the way.

If you read my Growth Strategy document, I would love to hear your reactions to it. And, if you run your own business, I highly recommend considering Matilda for brand strategy needs.

—Jono

2

Broken Growth

Grow your solo business this spring 🌷Save your spot

Read more

3 months ago · 13 likes · 2 comments · Matilda Lucy

What does security have to do with drawing on the web?

Two.js runs entirely in the browser, so I had not historically framed it as high risk software. But during the workshop Licensing and Security Compliance, I learned that your audience can also be an attack surface. Many developers who use Two.js are newer to programming, and security issues often arise not from direct attacks but from misunderstandings between a developer and an API. In reviewing the codebase with automated scanning tools introduced in the session, I revisited Two.js’s SVG interpreter, which renders markup into the DOM using innerHTML so the browser can parse it. It is practical, but it is also a dangerous parsing pattern if misused.2 There has not been an incident to my knowledge, but the possibility itself is instructive. I am not interested in restricting expression simply because misuse is possible. What matters, however, is transparency. If something carries risk, that risk should be visible and documented. This way the developer using the library can decide for themselves and make informed decisions.

You can never be too prepared

Another workshop focused on responding to security incidents, and the takeaway was less about fear and more about preparedness. When expectations are documented and a disclosure process is clear, project maintainers can operate with more ease because responsibility has been framed in advance. Before this session, Two.js did not have a formal security policy. Now it does. There is a SECURITY.md in the repository and on the website outlining how vulnerabilities can be reported and addressed.3 Nothing dramatic prompted it. But preparedness is a tool to use in an emergency and a tool to build trust with your community.

Here is how Two.js grew

Over the month I updated dependencies so the development environment reflects current standards, enabled automated vulnerability and dependency monitoring, formalized publishing workflows so releases are signed and traceable, and updated documentation to make potential attack surfaces explicit in public issues. None of these changes alter how you draw a shape or animate a scene, but they strengthen the scaffolding around the library. Looking at the project through a security lens also made me more aware of how durable early design decisions become. Like law in a democratic society, once a precedent is set, there is very little momentum to change it, and that durability carries responsibility.

Two.js can continue leveling up with your help

The cohort of maintainers in the program was generous and humbling, reinforcing how much modern software depends on open source and how much trust is embedded in that dependency. Two.js is sustained primarily through client work, and sponsorship represents only a small portion of its support. If it were better funded, I would focus on expanding its portability so the API can be used across a variety of environments and contexts; in mobile apps for instance. For now, the most meaningful contribution is participation. If you have thoughts on the new security policy or how risks are documented, I welcome that conversation on Github.4 That month I was reminded that maintaining open source is not primarily about bugs, but about community, trust, and leading by example.

See the other participating projects

I want to take the final moments of this reflection to thank Github and the dozens of people and partner organizations that make Github’s Secure Open Source Fund possible. I also want to thank the other maintainers and projects. It was so inspiring to see so many projects (many of which I have used in my own projects) and pull the curtain back in their own journeys in the breakout rooms. It was a truly global presence. My last thought on all this: security gets a bad rap. In my experience, security is an unspoken term that slows the velocity of building. I tried to avoid it or at least only engage at towards the end of a project’s lifecycle. The Session opened my eyes to how security can be a mindset. One that does not only protect the work you make, but challenge it to make it better.

—Jono

About this time last year, I started using Large Language Models (LLMs) like ChatGPT and Claude regularly for work. As a freelance type of collaborator, I found myself often in scenarios where I needed to get something done that was outside of my zone of expertise. I remember I used LLMs to build out header authentication for the microservices that support the Living Encyclopedia';5 definitely outside of my zone. This kind of need is too small to hire another person to help me out. In spring, I started using LLMs to help game plan projects: both client and personal ones. At the time, it felt like a kind of smart notebook. I could keep a tally of all my creative inquiries and have the LLM follow up on preliminary research for it. In the summer, I integrated LLMs into some of my source code repositories. This allowed me to asynchronously delegate tasks (particularly code related issues) to an LLM to do that I could then review. See the screenshot below. In autumn, I had LLMs build out end-to-end product roadmaps and then execute them. Again, the Living Encyclopedia became a testbed to develop a custom chat interface based on Matrix Chat’s protocol.6 Today, I use LLMs to offload a lot of work that I would have otherwise done myself. Tasks that would have taken a long time to do. AI is in my editors, connected to repositories, installed in my terminals, and I have AI apps downloaded to converse, search, and research for me. In all, I have spent north of $1,000 in the last year on various AI services and products. I have subscriptions and pay-per-use token charges by programmatically accessing AI services. They are mostly LLMs, but also educational material on how to prompt, and generative AI or GenAI services to produce images, videos, 3d models, audio, design artifacts, etc.. The landscape moves fast. And based on the initial boost I got last year, I have been motivated to try out services as they come out. I am constantly assessing whether a product or service improves or expands my workflow.

In a way, it is exciting. But, over the holidays it dawned on me. An unreleased client project I completed early last year is probably the last project I will ever write by hand again. It left me feeling nostalgic and sad. The way I had been working for the last twenty years had not changed that drastically. Then all of a sudden, that way of working ended. Without realizing it, I had offloaded a lot of choices. Were they artistic or creative choices? Or were they more minute and mundane than that? Is that even possible? Does it even matter? Ultimately very few people see or even care about the code.

In the New Dark Age, the author James Bridle describes a term in media studies and philosophy called the hyperobject.7 It is a system (technological or environmental) that is so complex a human cannot truly understand it or predict it. Think of a social media engagement algorithm or the weather or an electron. AI is one of these hyperobjects. And I have the AI make choices on my behalf. I am not sure it supports or damages my concept. I am not even sure I could explain how it could. It is so complex.

–Jono

There were numerous improvements that marked the demonstration of the new version of Living Encyclopedia. First, Research Mode was paired down from its previously technical presentation to look and act more like aforementioned LLMs like ChatGPT. Second, Create Mode was enhanced with a history of previous prompts. Dream Mode ushered in performance improvements. And finally, share functionality was added. If you had a conversation that you wanted to share with someone else or you created an image that you wanted to share, there were ways to save aspects of it to your device for later. Now, however, it is a simple click of the share icon and a permanent URL is created for you to share with anyone who has a web browser. Best of all, the recipient does not need an account to enjoy the content. These features comprise a more complete product experience for Living Encyclopedia. And it showed during Grand Ave Arts: All Access!

Try Living Encyclopedia v2

I had not shown work in person since the pandemic started. So, I got jitters the night before: part excitement, part worry, part pulling all-nighters in the lead up to the event. The event came and went in a blur. Over 2,000 people came through the Dataland pop-up space and tried the latest version of Living Encyclopedia. This included friends, family, and some familiar faces I had not seen in awhile. Rallying people around a positive and constructive experience, guided by art — this is what excited me about experiencing art. It is also what motivated me to enroll in graduate school in the arts. Dataland is officially slated to open Spring next year. With the reception we received during Grand Ave Arts: All Access, I could not be more excited to invite you to Dataland next year.

Wishing you warm holidays and an art-filled new year,

–Jono

Typatone, A Response

According to Wikipedia, Patatap was released on March 25, 2014. Thanks to the subreddit r/InternetIsBeautiful Patatap found its way to audiences I could never have imagined. I traveled to Europe and Asia to share my experience making it. For a lot of people, it resonated in a way that I was trying to communicate.2 It spurred collaboration and custom projects with brands and bands I admire.3 I made new lifelong friends. All great things. However, in making its way into popular culture, there was another theme that marked the reception of Patatap: procrastination.4 Numerous articles and as many email inquiries flooded my inbox to ask how I made something so addictive. Coming from a place of reverence for artistic craft and process, this honestly shocked me. Unfortunately being human, I had a hard time letting go of what felt negative to me. I did not want people to procrastinate. And I did not want people to be addicted to Patatap. So, when Shawn of Lullatone pitched the follow up as an experience to type and make melodies from words, I thought long and hard. I took walks. I rode my fixed gear bike. I went to parties. I tried to gain some perspective. And, in a small room of the Ace Hotel in Midtown Manhattan while on a work trip, I started prototyping what would become Typatone.

My feeling was that the “addictive” element of Patatap were the arresting animations. When you type on the keyboard, you do not expect the screen to flash. I intended to subvert user expectations, but only to make the experience delightful or surprising. If those were themes that marked Patatap, then with Typatone I wanted the themes to be subtle and meditative. “Could something subtle even be appreciated today?” was a question that ran through my mind. Remember this was when Vine content was capped at 6 seconds. I asked Shawn if the sounds this time around could align with these desired themes. Could they be more focused, narrow? It is why most of the filters are pretty slow. The animations in Typatone (compared to Patatap at least) are also subtle. There is the sing-along style identity mark that bounces to whatever letter is triggering a note place. There is also the backdrop, a linear gradient representing the filter. But, it is also representing the time of day. Even the keyboard, which is an intentional UI element, shows expectation, but is mostly utilitarian in behavior. Lastly, the sound clip composition is a mix of Lullatone’s sensibility of melody mapped to Letter Frequency’s most common letters in the English language. These decisions were meant to create Typatone as a more purposeful, smaller, yet hopefully equally impactful experience as Patatap.

The results ended up being similar. After ten years of both apps runnings, it is safe to say the receptions were similar. Typatone, like Patatap, was met with a lot of delight. I get emails every month about how Typatone is used in school settings. Despite this impact, Typatone never received any critical acclaim. In an effort to justify my decisions, a year after Typatone’s release, I published an article describing the potential academic value of Typatone.5 In all, the disappointing results led me to apply to graduate school; to better understand how my work could fit in with art, games, and design.

The Next Ten Years

Looking back to my thoughts and my writing from the mid-2010s, I saw a lot of frustration. Today, I see now, that it was that frustration that was stunting me, not how people received it. Over the last four months I spent time updating the codebase and going over what I had created. My assessment today is much more lenient and positive. I am thankful that the app still runs! But, there were numerous changes that needed to be made in order for Typatone to endure for another ten years or more:

1. The technology stack was about to break. When Typatone was made, I implemented my friend’s Backend-as-a-Service product called Parse. Parse still works great. But, it interfaces with a lot of services that were once included with Parse’s for-profit services. So, I moved all the databases and created a unified API that all live in the same Google Cloud project where the application and CDN are hosted from. This reduces cost and complexity. Unfortunately, previously stored messages will not be backwards compatible.

2. This update does not, however, eliminate costs altogether. Like Uber ten years ago, Cloud Compute was practically free. Now, running Patatap and Typatone together runs me about $250 USD a month. Not much, but not nothing. For me to provide these applications “freely” (i.e: without advertising) I have always wanted to turn Typatone into a freemium model.

3. In order to present a freemium business model, I want to bring value to the customers who will be spending their money on this. After 5 years piloting the download-your-message-as-an-audio file feature Typatone, I am pleased to announced that this will be permanently supported starting with a price of $1.99 USD per message. I hope you will find this to be a straightforward transaction.

4. This price not only supports the hosting costs, but insures that whatever you purchase can be securely and permanently stored on my CDN. Now, you get an email receipt upon completing your purchase. The email includes a download link that is a permanent part of the project.

5. Despite these additions, I made considerations to keep Typatone anonymous. That means you do not need to login to any Typatone authentication or Jono LLC systems in order to create messages or use the application. This feels like the right balance of freemium for now. (Sidenote: if you are interested to hear me talk about the possible business models I considered, this post from last year was my thinking on it)

Try the refined Typatone

Looking back on the last ten years of Typatone’s life has been a rewarding experience. The project has positively altered my professional and artistic path. I learned a lot. And I made a lot of friends along the way. I am proud to have had the project run for as long as it has. I am even more excited to keep it going.

—Jono

I am the kind of developer who does not adopt frameworks, libraries, or plugins at the speed they are released. I am also someone who makes their living out of creating “unorthodox user-interfaces” for the web, not building the most common ones. I initially did not see the need to take on building out a React connector. I thought if someone else did, that would be great for the community. In 2021, this perspective changed.

As I dug into the meat-and-potatoes of running a solo studio, I found myself needing to build a lot of “orthodox user-interfaces” for clients. Buttons, drop down menus, sidebars, navigations, toasts, marquees are some of the components I was making over and over again. They were still in service of unorthodox user-interfaces, but I needed more and more of the common stuff in order to frame the unorthodox, the weird, the unexpected. All my clients wanted their work built in React.1 They also, more often than not, wanted me to add that special something that Two.js made my work famous for. A couple projects I published on Soup to Software are attempts to write, manage, and build out Two.js interfaces within React.2 I’m proud of these projects, but I hit a ceiling. I learned it was complicated and cumbersome to pass around state information of Two.js within a React project. As you can read in the aforementioned posts, both projects I wrote about are single page applications (known as SPAs). They run on a single page without any user authentication in part because of the challenges managing Two.js within React presented. There had to be something I could do about this, right? The needs of applications and application development today continue to lean on complex scenarios of state.

In order to fully assimilate Two.js into React, there were two major hurdles to overcome. The first hurdle, React is technically a project made in JavaScript, but its true ubiquity lies in its interoperability with TypeScript. TypeScript is a superset of JavaScript. It is more strict and as result it is more predictable. React relies on numerous advancements by the TypeScript community to infer developer intent when building applications. Some of these advanced features are tab completion or module imports and reference listings of properties within code editors. These features make writing compliant code easier. Unfortunately, Two.js takes advantage of many of quirks of JavaScript. This is more expressive and less predictable (for a code compiler). This made it hard to make Two.js conform to TypeScript definitions. Type definitions or signatures are the way a TypeScript interpreter can do code inferencing while a developer is writing their code. To reiterate, my first hurdle was to make Two.js TypeScript compliant. It is still an ongoing project and it has taken me about four years to manually write the type definitions for Two.js.

You are probably thinking, why not use AI to do this? I tried many times. It always failed. Because Two.js leans on the quirks of JavaScript, it does not cleanly map onto TypeScript even though TypeScript is a superset of JavaScript. I ended up writing it myself manually; 6,000 line treatise of Two.js.3 What I found through hand writing the definitions, while slow, is that I am actually able to bring the quirkiness of Two.js to TypeScript. It is not fully complete, but it is usable for the majority of use cases I see people use Two.js for.

With the TypeScript definitions in a stable form, it became much easier to write a library that bridges React and Two.js. This is the new library I am announcing today: react-two.js. It takes the type signatures of Two.js and synthesizes it into artificial React components so that developers can declare Two.js objects as if they were declaring native React components. By bridging the state management between the two distinct systems, react-two.js unlocks the ability to build new kinds of interfaces.

Visit react-two.js site

Buttons with particles, wireframes with form input fields, animated titles like you would see in a film. Each of these is now possible within a React environment thanks to this new bridge. This first version allows for drawing all the primitives natively in React. You can easily hoist them in a parent component to modify as desired on state change. Lastly, it provides React native ways to define multiple Two.js canvases on the same page. There are still many features missing, but full access to Two.js is always present for more advanced use cases. I have already started using this within client projects and it has sped up my workflow tremendously.

Like all my open source projects, it is free to use under the MIT License. If you give it a try, I would love to get your feedback,

–Jono

What Changed?

First, the dataset is much larger than the previous one from 2023. Instead of a thousand artists the amount is closer to a hundred thousand. The script I wrote to get this data processed for three days. It pulled all the artists from the Artsy API and then also got their related artists. From these two raw datasets, I created a new single set of data that is compatible with the way I am visualizing the relationships. This new dataset follows the d3 format for force directed graphs, a common industry format for assigning relationships to be displayed in a graph.2 While it is based on this convention, d3 is not able to handle this many points. This is where the next major change is between the previous visualization and the current one. I developed a three-dimensional version of a force directed graph. Basically a way to calculate positions, velocities, and forces of related points on the graphics card. This enables thousands more points to be drawn and connected in real-time over any other publicly available method. The library, called Force Directed Graph3, allows anyone to render 3D graphs by simply supplying the formatted data.

The Results So Far

Visually it is much more celestial and less like pollen on a flower. There are clear clusters of different artists who all relate to each other. But, there is also a lot more noise. The previous interface, which allowed you to move the camera by dragging your mouse or finger, is now more cumbersome, slow, and inaccurate. Also, the performance on smartphones needs work. While it renders all the points, there is noticeable lag and even hang that spills over to the operating system. This is a big no-no for usability.

You can try the prototype by hitting the button below. In addition to being able to move the camera by dragging your mouse or finger across the screen, there is a panel you can open in the top right-hand corner. Upon clicking, you can change the parameters of the physics simulation. To me, this is where the power lies: to plot this many points and then change aspects of the entire system is certainly satisfying. It is also where I think the utility, story, and purpose of this prototype will ultimately lead.

Try the prototype

If you find this version interesting, I also made another version that lays out the clusters in a different way (link). It is more performant for mobile devices, but there is even more noise in the distribution. This is because the forces are modified and ultimately reduced by a large factor. In the first prototype every point has three forces: attraction to its related points, repulsion against every point, and a general gravity to the origin of the simulation. With the right settings this motivates the system to resolve its orientation where these competing forces create an equilibrium. It also creates compound associations, which really burn computation for devices. In the second prototype, I make the attraction to each link only go one way. This improves the performance and also creates constant movement. Like a game of tag, there is one point running towards its relatives. But the relatives are still running away!

I still do not know exactly where this will lead, but am bolstered by the progress nonetheless. The shape of art it continues to be something I think about,

—Jono

This was the vision that was presented to me with a logo mark and some other design explorations about how to visualize this. Despite initially being averse to sound on websites, the client came around when I put together a polyphonic prototype where a note echoed for every vertical line intersection that was passed by the mouse or finger on touch. The rippling visual effect, which is actually just vertical lines connected to springs, resonates with the echo of that specific note. The bottom-left corner represents the lowest order C and the top-right corner is the highest order B. All notes lie on the C major scale giving the sound space cohesive structure. The lines are drawn in SVG using none other than the library I maintain, Two.js and relies on masking, vertex manipulation, and layering unique to the library to get the final look and feel.

Play with the site

The end result is a tiny bit of music theory packed into a simple interaction with vector fluid lines and a layered grain. I find it subtle, but compelling to sit with it for a few minutes and explore the space. Hear the space. See the space,

—Jono

The core concept we looked to the video game industry is called juice1. Essentially, it is the confluence of feedback (visual, audio, etc.) at moments of player interaction to enhance the experience for the player. Many visual effects you see in games aim to serve this purpose. I started with adding particle effects to different parts of the experience. Below is an example of when you change an instrument. As you drag the object into different quadrants of the grid, the model (and audio which is silenced in this video) change. To enhance this moment, I made a particle emitter that creates white sparks in the direction you are moving. It is further highlighted by a one frame shot of a white glow. The visual contrast draws your attention to the boundary reinforcing the moment you change instruments. The team also added haptics so the phone vibrates a bit each time a change happens. The summation of these feedbacks add to the juice of the application.

Other effects add to the distinct style of the application, which sits somewhere between a traditional mobile application and a video game. Below I created an effect known as a shader that implements a domain warping algorithm. This technical speak means any video can have a textured look without impacting the device’s processing time too much. These types of effects are powerful because they have a small set of parameters (which I am changing in the video to alter the effect) that the team can use as they like. This effect can make many different videos (live-action, animated, slow, fast, etc.) feel in the style of Shibuya’s application. The versatility of the parameters and the uniformity the effect brings to different styles present more opportunities for juice. When should this video effect change? Like the previous example, it could change when you change instruments to further accentuate that moment. It could equally be used in a different setting to draw attention to a different interaction. Perhaps when you speed up or slow down the song.

Deciding when and how to link different effects to moments of interaction is an interesting rabbit hole for me. While there is not a right answer, the process deeply reminds me of two ideas. The first is the concept of transcoding found in media art practices which I researched extensively in my thesis during graduate school at Paris College of Art.2 The second is the design process for both user experience and branding. The Shibuya team spends a lot of time testing these builds with users to see if the experience is intuitive. Sometimes tweaking small things creates large impact (both positive and negative) with a test group. When users “get it” it is rewarding and enhances the feel of the game. But, this juice equally imparts a sense of what the brand or identity of the application is. Because we have decided what rules and moments are linked to interactions, we are offering our vision of what the brand is all about without needing a logo or traditional design element. Does the application feel playful? Serious? Neutral? All of these interactions reinforce an identity.

Try the app on iOS via TestFlight

As this is a progress report and not a completed project, the decision process is fluid. The amount of juice is all a consistent point of debate within the team. Regardless of the state, there is a lot of content on juice: talks, videos, books, articles. There are a few on how it intersects with design3. It is a fascinating concept that has resonant implications for the design process. It has been a joy to tap into the world of juice.

Here is to more juice.

—Jono

Before looking at this year’s numbers, were there downsides to a more balanced schedule? Yes. In July, I realized I was going to earn much less. This affected both my personal budget and my business. I had savings to use, but it did not feel good. I felt a wave of anxiety toward the end of the year. Money was tight, and I was searching for more work. I accepted some projects that I would not have taken if I was not stressed about income. Was it worth it for a great year of balance? Yes. I have no regrets. Could the end of the year have been less stressful? Yes. That is why I leaned into commercial work this year. If I front-load the income, I may reduce stress later. So far, I have earned much more than in past cycles. I am on track for a record year.

I got here by raising my prices and taking on more work. I increased my rates to match inflation. I estimate how many hours a project will take. I divide those hours by how many I will work each month. Then I offer clients a monthly retainer. This model is helpful for clients. It gives them a predictable cost and supports a more collaborative process. The price increase helped, but it did not double my income. The extra work did. As I said before, I took on unwanted projects when I was under pressure. To avoid that, I took a new approach this year. With support from my family and therapist, I decided to say yes only to work that interests me. I followed the energy of the moment. I also chose to trust in abundance. I did not expect this side effect: when the work is exciting, it no longer feels like work. This has been a big positive. I do not think I can maintain this pace all year.

The graphs and numbers reflect these choices. My personal work dropped from 37 percent to 11 percent. My sales and subscriptions also declined. This is not a surprise. I still maintain a few projects that receive occasional support. However, I have few chances to ask for donations or to charge for access. My website traffic is down. Also, there was a short time when my apps were removed from the App Store. That happened when I switched from an individual account to an organization account. These things explain the decline. I still have a long-term goal to rely less on client work. For now, that goal will wait.

The chart showing Commercial (blue) and Personal (green) work explains two key ideas from the last five years. In 2021, I wanted better balance between personal and client work. I thought that was the reason I ran my own business. This past year, I changed direction. I embraced the client work that came my way. With that shift, I doubled my income. That success brought confidence. That confidence made me more excited to work with my clients. This created a positive loop. At the same time, I feel like I am hitting the limits of what one person can manage. We will have to wait until next year to see how well front-loading income works. Until then, see you next month.

Review the YoY numbers

—Jono

Partnerships in the context of business boil down to value based transactions. I present something of value to you. You give me something of value in return. This simplistic view, however, does not reveal the intangible qualities of what a partnership fully entails. The personalities, contact, rapport, and goals of each party play a large part in how smoothly (or not) the transaction occurs. For as long as I have been building websites, I stayed within my expertise: the design of the page, the rendering of the components, the performance of the experience, the browser. So, when I started working on Dataland, I was upfront about saying what my expertise was and more importantly what my expertise was not: how to manage and sell a lot of tickets.

In early 2023, Refik Anadol, a founder and artistic director of Dataland, asked me to conduct research into ticketing vendors to support my gap in expertise. The team spent about 3 months interviewing and debating who to partner with before choosing. We then did not even start building the ticketing flow on the website until the following year. Below, is the comparison presentation between the two top choices in ticketing vendors I put together:

Check out the Ticket Vendors Comparison

When I made this presentation, I anticipated a three year contract. This is the standard initial engagement length for many of these white label services. But, this was an intellectual expectation, not an emotional one. I focused on the capacity for us to customize and tailor the user’s experience of purchasing a ticket on Dataland. I thought about the ticket price implications for the customer. I considered how and where the customer could purchase their ticket. To be blunt, I looked at it like I look at a lot of technologies: a logical transaction. Working with ACME has been much more than their API, customization tools, and volume experience. Each week I meet up with someone on their team and present the state of the website. They listen and along the way provide guidance. Yes, most of it pertains to how their product works. No, their product does not work 100% how Dataland would like to use it. As a partner, however, they are invested in the museum’s success. Their suggestions have as much to do about framing the ticketing experience as distributors as it does about using the ticketing platform. They helped me better imagine the ticketing journey.

In this way, they have filled many gaps in expertise. Their partnership has provided confidence as much as it has tickets. It has given me a new understanding of where business and arts intersect.

I am excited for you to use (and hopefully not really notice) the ticketing experience at Dataland,

—Jono

Soup to Software is a pun on the phrase “soup to nuts”. For those unfamiliar with the phrase, it basically means “from beginning to end”. Its origins stem from American traditions of multi-course meals where soup was often served first and nuts last. I like this pun because it represents me and my practice in a descriptive way. First, I relish in the tedium of handling each stage of my craft myself. Auteurs like surrealist director François Truffaut impacted my thinking about craft at university. I want to approach my work in the same way. Knowing firsthand how things work at each stage of a project is one way to honor the approach of an auteur. Second, I enjoy cooking. So much so, I often wear an apron when I sit down at my desk.1 The ritual of standing in the kitchen, chopping, assembling, and composing a delicious meal exercises many of the same mental muscles as creating a digital experience.

So that is the name change. But, why change at all? In 2020, I decided to wind-down my social media posts. At the time, I did not want to bolster platforms that were not aligned with my values as an artist. As time passed, it has become a simpler reason: I do not want my value to be contingent on social media. My process is slow. The experiences I make (hopefully) speak for themselves. But, they are not able to be directly experienced on social media platforms. My works are not videos or images. I use videos, images, and other source material to create my work. While the accessibility of social media is enticing and lucrative for some, a recap video or a swipe through of different images does not do me or my work justice. Writing in slightly longer format, however, does tap into the context concerning my work. So, a newsletter made sense to me. For the time being, so does cross-posting it to Substack.

To celebrate this name change, I made a simple “interactive soup” which you can see a video recording of me using the webpage above.

-Jono

The homepage has all the necessary information mentioned above. In addition to this, for visitors on their desktop and laptop computers, there is an easter egg that reveals what I call a mini Digital Audio Workstation (DAW). Shown above, the DAW Mini allows you to drag clips from Leila’s single and arrange them into your own eight bar loop. DJs and musicians often cut stems of a single to enable remixes. This interface takes it a step further by allowing anyone to make custom loops and cutdowns of Leila’s amazing samples from her single Incurable Dreams. The features of the DAW Mini include quantization, start and end delimiters, and mp3 downloading with metadata included.

Try Naz Hejaz’s DAW

Leila, a software engineer by day, built an easy-to-use and access Content Management System (CMS) to organize and store the audio samples. I added the interface on top. At the eleventh hour we decided to reveal the DAW Mini as an easter egg instead of as the first thing you see. One challenge making the DAW Mini was determining how many features to include. What has been great working with Leila is that she is a recording artist. She is the composer and performer, so the conversation of what was the simplest set of features opened my eyes to the perspective of a music producer.

The productive conversations and conclusions we came to make me think there is an opportunity (particularly with the not too long ago Epic Games’ sale of Bandcamp)1 to provide distribution tools that speak to music creators’ needs. Streaming is one of many things that artists and bands need to juggle when releasing their content.

Anyway, give it a try. I would love to hear what you make!

–Jono

In Dream Mode, we focus on a visualization of the AI’s mind (seen in the above picture as the celestial body on the right side). It is a way to spatially organize the data Refik Anadol Studio’s fed to the AI model. The user interface is reduced to a top bar to switch modes. While you can click, tap, or drag to pan around the space yourself, you do not have to. The application wanders through the AI’s mind highlighting points of interest and revealing different species of flora, fauna, and fungi. Refik has described it as “meditative”. Research Mode (seen above in its entirety), on the other hand, confines this visualization to a box. The mode populates many other user interface elements around it for you to explore. It is rigid in its design and generous in its presentation of information. Unlike current AI products out there, the Living Encyclopedia embraces two cultural ways of engaging with and consuming information: European design associated with minimalism and Asian design associated with high-density layouts.2 As much as I enjoy and sometimes identify with Japanese culture, I am not Japanese. So, while we took a high-density approach to laying out Research Mode, I relied on western ways of thinking to connect the boxes together.

The approach I took, is reminiscent of my 2021 piece Curaturae: Writing with Open Access. Every element in Research Mode has at least one interaction or reaction to another one. For example, if you click on a biome from the “Biome Index” on the left-sidebar it will populate into Large Language Model (LLM) chat box to the right of it. It also prompts the “Image Generation” to draw an image based on the biome’s scientific name. This kind of high-density layout with a tight web of correlated interactions has two academic references: Edward Tufte’s Supergraphic3 and Oulipian rules4. The result, I hope, is a kind of choose your own adventure through the data. If the chatting to the LLM resonates with you, then you can do that. If clicking on the weather data captured by Refik’s studio or server communications piques your interest. You can do that too. There is no right way. AI models are highly complex multi-dimensional systems. By bringing disparate aspects of the means of production to the user, our aim is add meaningful pathways through this complex system. So you can assess it for yourself.

More on Living Encyclopedia

Through different modes you have different ways to sift through the massive amounts of data compiled by Refik Anadol Studios. And through referencing different cultural ways of consuming information, we present a truly different take on AI systems. One made by artists, that I hope you will try. And starting today, the Living Encyclopedia is available to anyone. It costs $10 / month or $100 for a year of access. If you sign up for a monthly subscription you receive a 14 day trial. As a small gift to my readers, you can checkout with the promo code JONOFYI and receive the first month free (limited to the first 100 orders). To get started, click the red button above.

If you give it a try, please tell me what you think! This has been years in the making. These behind-the-scenes posts aim to convince you that this type of digital experience is not only unique to Refik’s artistic vision, but to mine as well.

–Jono

The video above is a one minute run through of the application’s Dream Mode: a lean-back experience that pans through the image latent space (or a kind of spatialized brain) of the AI model. It focuses on the overall form of the space and then dives into one point to focus, known as a label. Each label is a species of flora, fauna, or fungi found in rainforests around the world. The images and text are completely hallucinated or dreamed by the model. The studio has over several years built up this catalog and it continues to grow; hence the name Living Encyclopedia. It can run for hundreds of hours before highlighting a marker that’s been shown before. While the interface reduces the model’s multi-dimensional space into three dimensions, the same space is also seeded with sounds. As the labels change, so do the soundscapes that support it creating a kind of AI radio.

How is this part software product, part artistic gesture? The prior is straightforward. Once the Living Encyclopedia is available to the general public, you will need to register on Dataland’s website and subscribe (with your credit card) in order to experience it. This experience resembles other AI products on the market like ChatGPT by OpenAI or Claude by Anthropic. For more on how we are building it, you can click the button below to see our internal Product Requirements Document. Once in, however, what exists on the Living Encyclopedia is strikingly different from the aforementioned software products. We arrived at this interface because the team involved are approaching similar challenges to technology companies with an artistic framing. With it, we want to ask questions like how can this tool unpack the good, the bad, the misunderstood, in artificial intelligence? To frame these questions we relied on our collective knowledge of the arts. Personally, works of Doug Aitken3 and Tacita Dean4 came to mind. I wanted to give shape, pacing, and story to the world building process of interacting with AI models.

Read the PRD for Living Encylopedia

Unless you got into the beta access last month, you unfortunately cannot try this yet. But, when it does come out, I ask that you try to keep this in mind when using it. And if you do, please let me know your thoughts. In the meantime, I hope you can take my word that art lurks in software,

—Jono

The decision to focus on client services as an individual contributor was deliberate. It provided a foundation and clarified the kind of work I want to focus on: imagining interactions and building experiences people use. This decision did in fact open up space for the work. But, running this type of business requires many focuses: accounting, contract writing, and a lot of back-and-forth meetings to name a few. Sometimes I wonder if there is a more effective way to distribute my work and the work I want to make. Social media claims to have solved this. And recently, other ideas I saw online bolster social media’s claims.

In his talk earlier this year at TOKEN20491 Murad Mahmudov describes a compelling phenomenon. The speaker framed social media as an asset class for younger generations, especially those with limited access to traditional wealth-building means. This framing has stuck with me. It highlights a paradox: while I remain hesitant about serving my work on social media, I see its potential as a platform for wealth in a rapidly changing economic landscape.

I do not claim to have any answers to this, but this tension fuels much of my thinking as we move into 2025. What is motivating you as we move into the new year? Are there topics or themes you think I should focus on?

—Jono

You can read the official tagline and information about what this is over on the Dataland site (click the red button below). For those of you in the art and technology space, you will likely see Refik Anadol, the museum’s artistic director and cofounder, talk and share information about the Living Encyclopedia on social media. To me, the Living Encyclopedia is a compilation of data (images, text, statistics, video, lidar, etc.) Refik and his studio conserve and use to create his art. These data come from various sources and partners. And they center around nature. For the Living Encyclopedia, these data are presented in the form of part software product, part artistic gesture. The result is a state-of-the-art interactive website that lets you converse with an AI in only the way Refik Anadol could imagine. It is a rare attempt to imbue technology with the intuition of artistic practice.

There are three modes. Research Mode presents a chat interface not unlike ChatGPT with the Large Nature Model. It is surrounded by a visual of the model’s latent space and different data markers like weather, species, images, and sounds to guide the conversation in open ways. Chat Mode is a prompt box that generates photo-like images of both real and imagined species. Lastly, Dream Mode is a lean-back experience that is a fly through tour of the Large Nature Model’s latent space or “brain”.

Learn more about the Living Encyclopedia

Conceptually it ticks all the boxes of topics I am interested in and have been talking about in this newsletter over the last four years. Technically, it is the culmination of mastering the development of single-page applications. It uses contemporary build and distribution techniques in order to load millions of datapoints into your session in real-time. It renders spatial audio based on the markers in the latent space and continuously monitors memory load to run on mobile devices and desktop computers alike. The development time was not years in the making, but it has taken me my entire 20 year journey into computer programming in order to deliver the anomaly that is: part software product, part artistic gesture.

You can sign up through the link above. We will be letting in new testers throughout the coming days to preview the Living Encyclopedia before it eventually becomes part of the museum’s membership benefits.

—Jono

Before AI: a numbers game

Typically in the type of design and artistic experiences I create, there are functions that require a inputs. It could be weather data, sound input from the microphone, or a dataset from a government institution. Regardless of how this input is selected, it is assigned by a number at the time of use. For instance, the location of weather data is determined by two numbers, the latitude and longitude. The amplitude of the microphone is a number that is modified to create a visual connection between a circle and sound. A dataset has many columns and rows. Numbers define which column or row to display that data. It is why I like to think the French word digital is numérique. AI changes this.

With AI and specifically LLMs, I am not bound to numbers but to text. I send text to the model and it responds to me with different text. If you use AI models in your daily life, you likely write human legible text and in turn receive human legible text. This is an effective and relatable way for humans to search. While LLMs attempt to predict your response as text, they actually do not know (or care for that matter) the language of that text. So, you can send anything that can be converted into text. For example an image can be encoded into text through a process called Base64 encoding1. If I send that to the model, it will respond back with its own image encoded as a series of text characters. So, what used to be strict definitions of numbers now is a more vague, but broader means to select and generate data through text.

How do I use AI?

Speaking strictly to the artistic process, I use AI as a replacement for something undefined or ill-defined. Here is an example. In my piece, Nostalgia For A Past Future, from 2009 the center shape is created by thousands of dots drawn to the screen every second to resemble movement. Based on the mouse’s position that can look like rivers flowing or electrons jittering. This is created via a noise function, a kind of random value. Personally, I do not know nor care what the precise angle is for each element drawn on the page in order to create movement. I care what the movement evokes. So, I have offloaded that knowledge to a noise function. Today, I could offload this ill-defined value to an AI model. I can ask the AI model, “For this point N, can you give me the angle and velocity it should move in?” If you ask ChatGPT this, it will give you a long answer. But, if you wrap your question with additional text and feed that to OpenAI’s API, you can restrict it’s output to only give you what you need. A couple of numbers to use to draw the next point in the list. Interestingly, I can further elaborate my question with style; like, “imagine this point moves like a river.” The effect is roughly the same, but the process is totally different.

I am still finding ways to apply this line of reasoning in my creative process. I will use it to fill in copy for a website I am building, or to generate an image that supports the main element I am creating. When working, my creative process gets held up in the question: how should I make this? AI has become a quick-and-dirty way to get a rough response and continue working.

The pros and cons of AI

As previously mentioned, I can use these AI APIs to continue working through the bigger picture of an idea while not getting bogged down by the “how to” of a specific detail. This can be beneficial while on a time crunch. But, it also absolves myself of thinking critically. Sometimes, the critical thinking is the key part to realizing the work. I worry that my laziness could get in the way of discovering a true gem. Then again, it can save me and has saved me a lot of time. I am conflicted.

To the credit of the LLM makers, it is extremely easy to drop in a prompt to almost every step in the creative process of building software. This allows for abundant opportunity to explore and test the efficacy of an AI in ways I could never use a noise function. The potential of this is incredibly compelling to me. However, in practice, LLMs are so complicated and layered that it is often difficult to know why or how it got to the answer it did. As someone who relies on critical thinking to make my work, this uncertainty diminishes its potential impact. Again, I am conflicted.

Lastly, I need to take some data science class to learn how to properly tune these models. I say this, because they frequently hallucinate. When working with an AI, I imagine they are an actor in a film and I am the director. I will pre-prompt them to be a certain kind of character. And they always break character.

What I can show

These findings may be interesting, but what can I show you today? Unfortunately, not a lot. Most of the work I have done is not in a state that can be shared. But, what I can say is that most of the work you will see from me in the future will have AI woven into it. It will not be the focus, nor will it be overt. But, it will be there, helping me flesh out an idea. Defining something I am trying to communicate incrementally more clear.

Until then, here is a prototype I made of an anime chatbot you can play with. It uses the same technology that powers ChatGPT and presents it in a game-like scenario. This is for a client (scrubbed of identifying and proprietary information) and weaves in some interesting animation technology from Japan with the Web Audio API to present a generic non-player character (NPC) you might find in a game, but totally unscripted.

Try the AI Chatbot Prototype

How are you using AI? Do you use it to support the creation of art? Or is it itself the art? Curious to watch where this all goes,

—Jono