Areas of interest (June 2023)

Developer playgrounds, design-to-code automation, agents as users...

I’ve rounded up a few areas of interest that have captivated my attention in recent weeks. If you’re working in any of these areas or have strong opinions on anything I’ve touched on, I’d love to hear from you at [my first name]@upfront.com.

Design-to-code automation

As LLMs have proven adept at code generation, I’ve started thinking about how this might change the process of translating designs into working code.

In its simplest form, design-to-code automation might just mean turning design assets into markup. I would bet that Figma releases something like this by the end of the year (perhaps even this week at Config).

A more complex approach to design-to-code automation would involve keeping design and code components in sync with each other. You could imagine that changes to design components in Figma might actually be reflected in code–as JS components if so desired.

In theory, this would relieve engineers of having to manually keep code components in sync with design updates. But the reality of automating this workflow would entail a number of thorny challenges. A non-exhaustive list:

• How can you make sure that you leverage existing code components where possible?

• How do you ensure that generated code components account for all anticipated visual states?

• How should code-generation for UI components handle props and inline-logic, especially where it doesn’t pertain to visual matters? 

It’s possible that this kind of automation is more trouble than it’s worth. But I’m inclined to bet that a startup can figure something out here. If they did, I could see them owning a critical piece of the development process. 

Developer playgrounds

I really love playground apps like Jupyter notebooks, Replit, Gradio, Streamlit, and most recently val.town (probably one of my favorite startups of the past few years).

As I wrote earlier this year in Creative Machines:

The magic of playgrounds is that they miniaturize the atomic unit of creation. JSFiddles are miniaturized apps. Jupyter cells are miniaturized scripts. When the atomic unit of creation becomes smaller (like tweets vs blog posts), there’s a lot less cognitive friction involved in the creation process. Playground apps will likely be very attractive to users who find themselves increasingly creative and eager to experiment with new capabilities.

I believe that LLMs are going to make it significantly easier for people to write software. For the skilled developer, I expect greater output. For the novice, I expect that programming will seem increasingly accessible.

No matter their skill level, I expect we’ll see a surge of interest in writing software. As playground apps provide the fastest time-to-value for turning ideas into software, I think they will play an important role in the future of LLM-assisted programming.

Treating agents as users

Is it a good idea to anthropomorphize agents as a new kind of user? I can see at least one very useful reason to do so. If you treat agents as users, they can retro-fit into old workflows. 

For instance, if you wanted to assign work to an agent, you could do so the same way you would for a human colleague. Or if you wanted to ask a question, it would be nice to be able to do so in all of the contexts that you might with a human (in Slack, in Figma, in the comments of a Github PR etc…). And so forth…

One intriguing side effect of treating agents as users is that it enables what I see as a novel formula for apps working together: an agent that knows X kind of information about your business is able to leverage that knowledge in various app contexts.

Examples:

• An agent that knows your code base can help draw up technical plans in Notion.

• An agent that knows customer usage history can help draw up responses in Zendesk. 

• An agent that was assigned an engineering task in Linear can generate a pull request in Github.

• An agent that knows your business metrics can help write strategy documents or create presentations.
  

A question I care a great deal about is who will end up building these agent-users.

Will the agent that you dispatch engineer work to be provided by the same company that supplies the coding agent that your team uses in their IDEs? Or will this kind of agent come from a different provider?

Will the agent that helps your team respond to customer tickets, using the knowledge it possesses about your customer data, come from your customer support tool? Or will that agent be molded and administered by something else? 

In a world where the agents we use are not staffed by the applications that host the main theaters of our work (i.e. incumbents), I wonder: where will they come from? How will we train them? What will we use to teach them the things they need to know?