About a month ago I asked a friend if he was using any automations as part of his daily work. In response, he said he hadn’t used much AI other than the obligatory chatbots, and that he felt behind. It made me think about what I take for granted in my own AI use, and really ignited my curiosity about how everyone else is using AI.
As a designer, what do you do in these circumstances? You ask people. So, I made a survey on how people use AI at work and how they feel about it. This survey was sent to friends and a few other social and professional groups that I’m in.
107 people took the survey. High level findings: people are using a lot of AI at work, and AI replaces or augments the main work that they do. Respondents like it, and most surprisingly, software engineers who use AI to do a lot of their core work still think using AI makes them more valuable. One caveat is that these respondents are AI enthusiasts, so if anything these numbers are the current ceiling, not the average. Here are the full results:
A note on methodology and my own AI use
This being a survey about AI, I wanted to be really cognizant about how I use AI in this process. Overall, the start to finish took over a month. A few days to write the survey. One week of procrastination. A little over two weeks of result collection, and about a week of result analysis.
The initial survey was written in conjunction with Claude. I penned down a very rough first draft, fleshed out the structure with Claude, and iterated question by question until we got it tight. I knew from the beginning that the survey needed to be 15 questions or less, so the main decision points were making sure each question pulled its own weight, and balancing the types of questions so I got both comparable data points and rich insights.
An important precursor to analyzing the results was registering my hypotheses. I knew that once I set eyes on the results my “findings” would be colored, so it was important to figure out what I expected. From the initial survey I got Claude to ask me a bunch of questions about my expectations. I answered each one of them and registered what I thought the answers would be. The results turned out pretty much in line with the hypotheses, differing in magnitude, with the exception of the finding above where the people who do their core work with AI still find that using AI makes them more valuable.
Analyzing the results took a few passes. The first was making sense of all the quantitative responses. My tool of choice for this is Jupyter notebook, and I used Codex and Copilot to help build most of the notebook out. I first graphed out every question, then broke each one down by demographics.
Once I was satisfied with all the quant data, I then took a pass on the qualitative ones. I first read them individually to get a sense of what the overall picture looked like. Then, with Claude, we coded the rows together. This was an iterative process of asking Claude to come up with the code book, me going through row by row to see if it fit, and iterating with Claude to refine both the coding and the code book. I thought this would be easy with AI; it was easier, but it was also tedious. To finish off, the transformed data was brought back into Jupyter and graphed.
When everything was done, I looked through all the graphs and wrote down everything that stood out to me. After that, I handed an AI all the data and my findings, and fleshed them out. I ended up with about 3 pages of bullet points.
To construct the final deck you see above, I first highlighted the most compelling findings and then worked with Claude to construct a narrative arc for a presentation. The deck was created in Google Slides by hand, alas. The charts were created in Google Sheets with data extracted from the source worksheet by Cowork.
It was fun, interesting, and ended up being a lot more work than I had expected. I would say that I benefitted immensely from having AI as a collaborator on both the thinking and the tedium of this exercise. AI is not always or even often right, but by having it give me something to edit and start from, it has made it easier to keep making progress. Without AI, I wouldn’t have been able to achieve the amount of rigor, and frankly just spend the amount of time and effort on a project like this.
So was my friend right about being behind? Depends on who he compares himself against. Yes if it’s against the bleeding edge of AI adoption. And an empathetic no, because managing humans is not something AI is good at.