Posts

User testing challenges: Ways to test microsites without the secret getting out

New attractions are widely anticipated throughout the amusement industry. They frequently come with changes to a park, zoo, or aquarium’s website. This could range from a mere mention of the new attraction to an entire website (microsite) created just for building anticipation for that new attraction.

How do we do usability testing in this situation? Do we have to wait until the microsite is live before we can test it?

No. There are several possible approaches.

3 ways for amusement parks to test microsites in progress with users

1) Non-disclosure agreements

One possible approach which requires less design work to be done is a non-disclosure agreement. Test participants are asked to read and sign the agreement before they can take the test. The agreement keeps them from doing the following:

  • discussing the fact that they tested your site with anyone else
  • discussing anything they learned in the test (e.g. about the new attraction) with anyone else

User testers normally also have to either sign a consent form when they do a study, or agree to a third party platform’s terms of use if the tests are conducted through a third party. The consent forms say how their test data will be used, what the session will consist of, and that they can cancel the test at any time.

2) Use alternate copy

Test sites can use alternate copy that obscures the real identity of a park, an attraction, and its related companies. This can be lorem ipsum placeholder text or text that refers to a generic attraction at a generic park. Alternatively, a test site could use copy and images related to a previous new attraction at the same park. User testers are always told to assume a certain test scenario when a test starts.

3) Make another site in parallel, and do the user tests on that until public launch

Some parks create viral marketing campaigns which lead the public to believe that they will add one type of ride, only to announce another. Similarly, if alternate copy is not enough, we can take a combination of these measures:

  • Add a non-disclosure agreement.
  • Password-protect the test site.
  • Hide the test site from search engines.
  • Create a second test site with the same design style but different copy, images, and (if applicable) videos.
  • Do the user tests on the second test site until the real attraction is announced and the real microsite launches.

Have us test your site with users

Want us to look at your website, test your site with users, or do a redesign? You can now schedule a free project consultation with us via our YouCanBook.me site, or contact us via our site to get more information. (An initial consultation is free. We sell our evaluation services separately.)

Getting user testing ROI in the amusement industry

Last week, we mentioned that hundreds of users provided feedback into our recent rebrand. They gave us input on company name ideas, tested first impressions of our site, sent us back to the drawing board many times, and walked through common scenarios to tell them what was and wasn’t meeting their needs. And our site became all the better for it.

An important metric in analyzing web traffic is bounce rate: how many users enter a website and leave it rather than going on to different pages within the same site.

Guess what our company’s site’s bounce rate is over the past 10 days – since several days after our site went public?

Zero percent. For hundreds of page views by people other than us. You can’t do any better than that! (Edit, 10/23/2014: While this was only for a 10-day span, the average bounce rate for a website is estimated at 45% to 50%. Ours still remains much lower than that.)

So we must be doing something right. And we’re ecstatic that our site is giving you something of value. In this post and our next one, we’ll be describing the benefits of user testing and ways to do user testing on a microsite without giving away your big announcement.

What user testing gives you

1) It tells you what users really think of your site – beyond the metrics.

Analytics will tell you how long people are staying on your site, what pages they are visiting, and if they are converting – among other things. But you may have created a good site or a site that people hate using, and they could look the same in analytics.

In a usability study, users think out loud to tell you what they really think. And participants are compensated, so they don’t rush through a well-written study like they do on free website surveys.

2) The feedback allows your site to improve your business’s key performance metrics.

Most people who would leave your site without buying anything will not tell you why. Users who walk through scenarios on your site are able to tell you what confuses them, frustrates them, or would cause them to leave your site.

If you hire us or an outside firm to address problems that users find, and then you address these items, future visitors to your site will be more likely to buy from you.

3) It helps your users to be less confused and lowers your support costs.

Sometimes, users have already made up their minds when they want to purchase from you. But something on your site confuses them and slows them down or stops them from purchasing. So they decide to contact your support team.

Fixing issues found in usability testing will allow your users to be less confused. That will get them through your sales funnel faster and keep them from needing to contact support. And that will allow your company to spend more of its budget on the products and services that make it great.

4) User testing gives you an advantage over your competition.

From what we have been able to determine, most companies in the amusement industry are not doing usability testing on their digital products with users outside their organization. Although some park chains gain much of their competitive advantage via user experience design in the physical world, most companies in the industry are not implementing UX design methods or user-centered design processes. Many amusement sites still cater to very feature-driven enthusiast market segments. Developing a site with great features is enough for many of them.

But the most successful websites today have invested substantial time and effort into creating the best user experience possible. Name any wildly successful online startup, and they have done this.

Users’ expectations are now changing to the point where they take startups’ strong investments in user experience for granted. Introducing usability testing on an existing site is a great way to start seeing the benefits of this way of thinking before investing in a user-centered redesign.

5) It lets you use development time and budget more effectively.

Imagine that you and your team have several months to redesign a website and you have not committed to usability testing. You hand your team a list of requirements, and they get to work. They design. They write code. They test the site on all sorts of devices, and finally they deploy. Launch turns out to not give you the results you were looking for in metrics like bounce rate and conversions. And it all turns out to be due to a flaw in the site’s design. You tell your team that the site needs to be changed.

Let’s assume that this change requires significant layout and interaction changes to a key page in your site. At this point, the developers on your team will most likely be pushing back. Weeks or person-months of effort will have gone to waste because the site was designed and built the wrong way.

This wasted effort could have been avoided with usability testing. A card sort study would have shown problems in the site’s navigation before any of the site was built. A test on a prototype of the purchasing path would have revealed reasons why people would abandon their cart before any code was written. Big problems can be solved in hours instead of weeks or months.

Usability testing, therefore, confirms or denies that you are on the right path with a design in progress. And if you’re on the wrong path, it is much easier to get on the right path if less effort was spent moving in the wrong direction.

If your users can’t see your work in progress yet

All of this is fine and good for most situations that parks or ride companies face during website design and development. But what about microsites for new attractions – where we don’t want the public to know about a new attraction until the big reveal? We’ll discuss that next. To find out about our new posts, subscribe to our newsletter (at the bottom of this page) or follow us on Twitter.

Have us test your site with users

Want us to look at your website, test your site with users, or do a redesign? You can now schedule a free project consultation with us via our YouCanBook.me site, or contact us via our site to get more information. (An initial consultation is free. We sell our evaluation services separately.)

Our company's mission is to design delightful digital experiences for purveyors of joy.

How the Thrill & Create site was built

Recently, I made several big announcements on this blog. One was that the company is now called Thrill & Create LLC, and another is that the company’s new website is now online at AmusementUX.com.

The previous site

The initial work for AmusementUX.com, internally abbreviated AUX, started almost two years ago. The company’s old website, at DalandanConcepts.com (pronounced “dah-LAHN-DAHN”), was the first new website I had designed or developed in about 9 years. (I had built several music-related websites, which are mercifully no longer online, over the 5 years before that – using a very old Mac version of Dreamweaver.) The Dalandan Concepts site was entirely hand-coded HTML and CSS using the now-deprecated 960 grid system. In other words, it was intended to only look good on desktop computers: not a good selling point since I was trying to sell usability consulting services using that site. Last year, I would take part of a weekend to make the site responsive so that it would be at least somewhat usable on mobile devices.

Preparing for Project: Amusement UX

I started planning the site’s replacement almost as soon as I launched it. While researching hundreds of amusement-industry professionals on LinkedIn, I generated a persona spreadsheet. This was relatively preliminary user research compared to what I do now, but this spreadsheet informed the design of the rest of the project.

I initially knew the site as “Dalandan V2” (version 2). I began by developing a desktop-first wireframe and later replaced that with a mobile-first wireframe. Fortunately, I was then busy with client work for a while. Many incoming phone calls during that year, in which people mispronounced “Dalandan” showed me that Dalandan was not a good word to use in the company’s name, whether I had nice pictures of dalandan (a type of fruit which I ate in Southeast Asia) to use for the website or not. Ultimately, what the site needed to sell was design services, not food.

So aside from making the existing site work on mobile devices, buying AmusementUX.com, and identifying a WordPress theme, I did not do any additional work on replacing the company’s site in 2013. I decided to shelve the project until mid-January 2014. Another big decision was to use a daily Scrum process (adapted for a Scrum team of one) in order to design and develop the new site.

Amusement UX: design and development in full swing

The project’s overall structure, including both design and development, consisted of some preparation work, 7 iterations, and 4 spikes.

Iteration 0

This was a brief iteration before work on the Thrill & Create site truly started. I installed a Coming Soon page and wrote some brief copy describing our business. Design work during this phase was fairly minimal.

Iteration 1

During this iteration, I performed my first round of ideation. Hundreds of ideas were then pared down to a much more manageable set that fit the site’s primary personas, and I did sketches and wireframes. My tool throughout the wireframe/prototype process was Axure RP 7. I also wrote preliminary copy for several pages. On a client project, I would have wanted to do a round of testing with users at this point.

At this stage, the homepage was much longer than it is now. Services, blog posts, and the About section were described on the homepage. An initial Process page, not yet reviewed with fellow designers, was already part of the site.

Iteration 2

Iteration 2 fleshed out more of the ideas for the site’s Services page, Process page, and the homepage. The homepage, at this stage, really aimed to establish the site as an authority regarding usability and user experience. It also did more to sell users on responsive design.

By the end of iteration 2, I had made medium-fidelity prototypes of most of the pages in the site. I had also run through Zurb’s Design Triggers list and incorporated many of those ideas. This iteration ended with a round of short usability tests to gauge users’ first impressions of the site.

Iteration 3

The first impression tests told me to revisit the layouts of the homepage and its hero area. I generated 4 hero area ideas and 8 homepage layout ideas and created wireframes and medium-fidelity prototypes of each one.

Spike 3.5

I then ran a survey wherein I paid many users to tell me what homepage layouts provided the strongest, most professional first impression. I chose pairs of ideas to compete against each other in this. Two out of four pairs did not have a clear winner, so I created layout ideas 9 and 10 as hybrids/replacements of these 4 other ideas.

Iteration 4

Iteration 4 incorporated feedback on the hero area surveys and a next round of homepage layout surveys. These resulted in some modifications to the hero area and the homepage. At this point, I ran a set of longer usability tests on the prototype and triaged their feedback. During this stage, I was also working on some business strategy options related to my usability evaluation service offerings.

Iteration 5

The longer usability tests gave me a wealth of valuable feedback. Among other changes, I continued to sketch new ideas for the homepage layout and created new variants of the hero areas. I also created new ideas for the Process and Services page, wrote their copy, and created two more comprehensive prototypes of the Why User Experience Design Instead of Web Design? article. During this stage, I was also making preliminary choices for the site’s typography.

Spike 5.5

Several users in this round of usability testing did not like the color palette which the site was using at that time. Since changing the entire color scheme of a site involves widespread changes and it is not (as of this writing) a simple process in Axure RP, I created a separate sprint spike to work on this. By then, the site was already well into its development process. The Axure RP prototype with the new color scheme gave me a reference for how the site was supposed to look after my code changes.

Iteration 6

Iteration 6 began in early May with adding content to the site, which was still hidden by the Coming Soon template. I started writing a custom CSS file, which eventually grew to well over 4000 lines of code. I spent most of this iteration working on the custom CSS and its associated work items. There was also a support issue with a vendor which took weeks to resolve. It pushed back the launch of the site due to the issue’s severity and the amount of development effort I had to expend to formulate an acceptable solution. Business owners wear many hats, indeed.

Spike 6.5

Spike 6.5, which didn’t meet a standard Scrum definition of a spike as well as I wanted it to, was mainly used for fixing bugs with the site which I had found on mobile devices and for starting trials of the fonts I was going to purchase, in advance of testing the site again with users.

In Spike 6.5, I created Axure prototypes of the homepage’s layout at more widths to show what a working version of the site would look like at those widths as I worked through the bug list.

Iteration 7

At the end of iteration 6.5, I conducted another round of long usability tests, including several tests with other UX designers. This, again, resulted in much valuable feedback.

At this stage I implemented an element collage to ensure a more consistent look and feel across the site. Element collages are now an artifact I produce much earlier in my design process.

I created five new hero area ideas in response to user feedback. The homepage, portfolio items, process page, and services page also received significant changes, for which I created sketches, wireframes, and prototypes.

In development, iteration 7  also involved making sweeping CSS changes because I changed the site’s font pairing in response to user feedback.

Spike 7.5

Spike 7.5 included more mobile bug fixes and many deployment tasks. This spike ended on August 21, 2014, when I soft-opened the site that you see today with a placeholder company name. After more branding surveys, I began the process to officially rename Dalandan Concepts to Thrill & Create.

Next on the agenda

Here on the Thrill & Create site, you may find information about who we are and what projects we are doing. Our Facebook and Twitter pages will have further, more frequent updates from the intersecting worlds of user experience design and amusement.

We would love to work on more user experience design projects and usability evaluations in the amusement space. Please contact us via our contact page or info@amusementux.com.