This is the third part of the third.
‘MVP’ challenges
We worked on Pinboard with varying degrees of intensity (at peak OKR was supported by three developers, sometimes it’s just me, 10% of the time I use hack days and other sassy opportunities to thwart this as well as other OKR work)… perhaps the time from conception to implementation across the entire organization This is one of the reasons why the period lasts more than two years.
Probably the bigger reason is the difficult situation we find ourselves in when trying to decide on an MVP for a project like Pinboard. Something like Pinboard lives or dies by its user base, which we call “critical mass.” If only a few people are using Pinboard, it’s not worth much, because the conversations won’t be that helpful… and they’ll probably stop using it… and our efforts will be wasted and potential opportunities will be wasted. So we were very conscious of this concept of “critical mass” and it was in our minds from the beginning. To make sure users want to use it, we have already added many features and functions that you would expect to be added many times after the MVP for other projects. It became very feature-rich and polished at the point of mainstream launch.
This is something we thought about in a mega retro we ran a few months ago, shortly after the enterprise-wide rollout; Here we invited all the developers involved in the process and the different engineering managers of the team during the time we invited. I worked at Pinboard. This point was brought up that this has suddenly become a complex chat app; We could do something simple in the tools to make it a little easier for the images to surface, which is a point worth bringing up, but in preparation for this post. I found one of the first OKRs to articulate the ‘opportunity’ of expanding the scope beyond a simple way of suggesting images. I think “opportunity” is a nice way to think about this and reassure us that we are doing the right thing.
Adoption
From the beginning, we remained very closely involved with our editorial colleagues, some of our early shadow work, but also with other more senior colleagues, showing prototypes and designs and getting feedback. Lots of meetings and demos.
As we got closer to “MVP” (albeit long-developed, feature-rich), we started experimenting with real users in production. Because Pinboard only installs if the user has permission, we can easily onboard users to launch an experiment and disable them later with the help of our Central Production team.
First of all, we made a few experiments with live blogs., We chose live blogs because they are relatively short-lived (typically no more than 24 hours) so we can collect feedback and make observations in a fairly short period of time. Overall this was successful, users saw the benefits and we gathered a lot of feedback to improve the product.
We then needed to test the value of keeping these conversations alive alongside the piece for a longer period of time, with people handing off over days. So we managed to run a trial with the global development desk and, more importantly, with all the people involved in the process across different functions (writers, sub-editors, art desk, etc.). This was again successful, users saw the benefits and we collected a lot of feedback to improve the product further.
Given some positive and tight relationships with the newsletter team, they asked for Pinboard early, and we let them, and we observed that they used Pinboard almost exactly as we envisioned and as users hoped (sharing and discussing images, discussing emoticons), delivering etc.) even if it disrupts the plot of a large portion of Succession for one of our developers.
Meanwhile, Ara was developing a great idea: an “interactive tour” that could guide users step-by-step through all the features. It even allows users to practice sending, mentioning, editing and deleting messages. When using Pinboard for the first time it will start automatically, but it can also be started manually at any time.
Thanks to successful trials combined with the interactive tour function, we were given the green light to release to all journalists. We wrote a tweak to the permissions mechanism to remove pinboard permission from Composer or anyone with Grid access and… It was live for all users on May 15 🎉. There are no production issues, no rollbacks, a slow but steady uptake (we observed from our metrics dashboard), and some good feedback via the dedicated feedback form linked from within the tool. The launch was a success, I put together a slightly scary ‘decorate yourself’ cake to celebrate the launch.
Here’s what we’ve seen since launch:
-
We have really engaged users in the editorial space (including seniors) who believe in Pinboard and encourage others to use it.
-
We continue to have really productive meetings with different teams, we’re making sure more and more team groups are available to speak to, our audience team and lawyers will be coming as well.
We saw a moderate increase in usage after:
-
A promotional video (similar to the video at the top of this post) that plays frequently on many TV screens in the office.
-
I also pitched the promotional video at the company-wide morning conference (suggested by one of our concerned senior users).
-
It is featured in the organization-wide internal newsletter through a “tell me” section that includes a link to the video.
What’s next for Pinboard?
While OKR hasn’t been a focus since launch, we’ve still managed to find time to implement some nice improvements to Workflow integration in direct response to user feedback from many users.
We made a start on the ‘important messages’ feature as a side project in sessions with someone I mentor. This aims to address one of the most egregious uses of our current tools; users leave important messages to each other using ‘note-taking’ in the composer’s main text, which risks being published and causing embarrassment at best and legal trouble at worst. We want to move such messages out of the main text and into Pinboard (where they cannot be published), but display them with the same visibility as notes at the top of a piece.
Another side project is that users can get images cut, retouched, etc. from the Imaging team directly via Pinboard. To make it easier to request. InDesign has a great printing process where users can right click on an image for paper and create a ‘View Order’; This appears in a queue directly in Photoshop where imaging team members can pick up and share the work. restore the edited image. The flow of doing the same for digital (rather than print) is terrible by comparison and involves emailing and attachments. This problem is described as follows: “It has not been resolved for 10 years”…so I look forward to the day when we can announce that a few clicks on Pinboard are all it takes for requesting users, and the same familiar and friendly queuing experience for folks on the imaging team.
This is the third part of the third.
Credit
Built by Pinboard Tom Richards, Jenny Graham-Jones, Thalia Silver, Andrew Nowak, Philip Barron & Ara Cho with additional developer contributions Fred O’Brien & Samantha Gottlieb. Product design Ana Pradas and Product orientation calvin dickson. During this time he is supported by the rest of the Content Production team in the Product and Engineering department. We should also not forget the contribution/time/effort of countless Guardian journalists who helped shape Pinboard.