About Smartsheet

Smartsheet is a collaborative work management platform of tools. What’s that mean? We build an extremely flexible and powerful tool that allows people to build business solutions based on their data. I liken it to a tool belt – There are lots of different tools at the users disposal to use in any combination to create something that works for their needs.
‍
At Smartsheet, there are generally three types of users:
‍
Solution builders - they have a process or need that they need to facilitate with software. That may include data capture, modification, reporting, automated actions, and sharing. The builder uses grid based sheets as the core of their solution.
‍
Collaborators - these users consume the solutions that the builder creates. They might complete data update requests, fill out forms, leave comments, or edit documents.
‍
‍System Admins - They set up the accounts, manage teams and roles, and establish enterprise policies.

Project background

The email customization tool is primarily targeted towards solution builders, especially those who use Smartsheet’s automated workflow capabilities. In most solutions that get built within Smartsheet, a core component is notifications of some sort, sent to various collaborators in one or more groups. Clear communication to the people using a process built in Smartsheet is critical, and having a tool to customize that communication is central to the solution builders’ experience.
‍
This project has an interesting origin. I served as a product design manager across two different areas (Automated Workflows and Notifications) that shared common customer journeys. After observing user feedback and usage data, I recognized that many pain points of our solution builders revolved around collaborator adoption. At the same time, the biggest pain points for collaborators came from poor email notifications.

I hypothesized that by providing better notification building tools inside the automation builder and elsewhere, collaborators would be more likely to engage with the process that builders were creating. I found a perfect time to explore my hypothesis during one of our company hack-a-thons. I formed a small team and after a week, I presented our work to the company and won a customer value award. The project was slotted into the roadmap, and we began to flesh out the rest of the details of interaction and capability as we kicked off the rest of the project detail work.

Gathering insights

The hack-a-thon team dug up historical research across several relevant feature areas to make the most of our initial week-long timeframe. We looked at past user interviews, concept studies, usability tests, product usage data, and feature requests to build up a list of insights. These insights provided the basis for much of how we framed the problem in a way that make it easy to break down and achieve success.

Framing the problem

Every feature that the UX team works on at Smartsheet starts with a “framing the problem” document. We use the document to:

Consider user needs as well as business goals
State our intended impact and potential effects
List our assumptions
Ask what we don't yet know
Define our context and constraints

While completing the framing of the problem exercise, I enlisted the help of a UX researcher, Austin Clark, as well as a senior UX designer, Vivian Stitzel.

The key takeaways we defined were as follows :

Process builders have little confidence in the emails that get sent out via automation, because they are not sure what they will look like, or what content will be included
Some email types have low open and response rates, especially when not customized
Builders were not able to test and verify their emails
Emails were missing critical configuration controls for some use cases
Analogous functionality had inconsistent interactions across the tool
Critical functionality that already existed was being missed due to poor visibility of power features

Defining our vision

After the framing exercise, we came up with a clear vision to guide us into the wireframing and visual design phase.

As we dug into the problem space, we realized that there were several areas that needed design improvements, too many to work on in one project. We used the framing the problem document to help us pick the most impactful scope to tackle first.
‍
Our overall scoped vision for the email customization tool was to provide an intuitive experience that highlights the customization capabilities users need to create effective, branded emails with exactly the right content.

Diverge and conquer

Because we had just one week to work on our initial vision pitch during the hack-a-thon, Vivian and I quickly sketched and came up with several different variations of the interface, each with a slightly different interaction or location.
We evaluated each solution and idea against our list of problems. We brought in two developers to give guidance on the technical feasibility of each approach that we had wireframed out. After considering each idea against those criteria, we put together a concept prototype to test with users.

Concept validation

We followed up our divergent design iterations with testing to validate and compare each design. We conducted a few iterative research activities: a concept test, followed by a survey, and then a usability study. We conducted the concept test and survey to deepen our understanding of customers’ email behavior, test the desirability of the advanced message customization feature, and to prioritize the customization options. We conducted the usability study to ensure customers’ ability to navigate to and use the feature.
‍
The concept test and survey findings reinforced the potential for this feature to help increase the response rate to Smartsheet emails. Customers reacted positively to the feature in the concept test (particularly to the ability to preview the message and apply custom branding) and survey participants reported both a high likelihood to use the feature and that they would send more emails from Smartsheet as a result of it.

Overall, customers were extremely excited about the idea of this feature being built

Design iterations and prototyping

After we validated our concept and knew we were delivering customer value, we held a UX review to bring the design work and findings to our executive stakeholders. We gained strategic alignment and moved forward.

This project was large, so in the UX review, we provided a step by step path to our north star vision, through incremental releases that would provide customer value while moving our feature towards the end goal.
‍
From there, we began to refine the design visuals and smaller scale components and interactions. We used NN/g heuristic research to help select the most appropriate component types. This project was also the second to move towards the model for interactive canvas style builders on the Smartsheet platform. The unification of the interaction strategy across pillar areas had the side benefit of making it easier for builders to learn how to build solutions in Smartsheet.

Usability testing

After the survey and concept test, we made a shift in interaction approach to respond to feedback. We decided to move to a more direct canvas manipulation interaction, rather than a configuration area and a preview area as separate functions. We followed up with a usability study to comparatively evaluate both approaches for the best task success.

During the usability study, customers could complete most of the tasks without any difficulty at all.
‍
The main area for improvement we uncovered had to do with discoverability as opposed to usability: Participants didn’t realize all of the customization options available to them until prompted to make changes.

The study also reinforced the high customer satisfaction with the feature: participants rated their overall satisfaction a 9 out of 10 on average.
‍
We made modifications where our research had found discoverability issues, by bringing forward the elements that were missed by using different visual treatments, and giving them more pronounced micro interactions.

Next steps

This project is currently in development, with some features in private closed beta. So far, those who have used what is already dev complete have reported that they are now far more aware (and likely to use) the email customization features than ever before. We worked with our product and engineering partners to build in appropriate telemetry to capture the data and metrics that will give us insight about whether or not we are solving the problems that we set out to solve.
‍
As this feature is released to a larger beta audience and early access, we have planned follow up usability sessions, where we will observe customers using the feature, and describe their experience. We will also follow up with a behavioral survey to understand at a larger scale the impact that our feature has made.

Advanced Message Customization