Session Notes, Sessions 7–9: Agile UX Summit, 2/25/12

Anders Ramsay: Learning to play UX Rugby

  • Feeding the beast
    • Whole team of devs and they’re building shit faster than I can design it
  • Half-baked UX
    • PO’s under pressure to deliver the next release and signing off on crap
  • Sprint-tunnel vision
    • Yes, we delivered all the features we were supposed to deliver this sprint, but the design is an incoherent mess!
  • Why the dysfunction?
    • Playing the old waterfall game on an Agile playing field without knowing it.
    • Lacking the thinking tools to understand how the Agile game is played.
      • Scrum is not an acronym; it’s related to rugby.
      • “stop running the relay race, and take up rugby.”
  • Relay race
    • Team members run alone
    • Collaboration isn’t built into the game; the baton is handed to the next person
  • Rugby game
    • Intensive and continuous collaboration is core to the game
    • Reach the goal line again and again to win the game
  • Rugby in design
    • Team communication
      • Workshops, not meetings
      • Intensive passing game across roles/perspectives
      • Iterating towards shared understanding
      • Move away towards “debugging,” more towards a collaboration-centered design process
    • Collaboration-centered design
      • Shift towards facilitation as a core skill set
      • Cardstorming
      • Collaborative Chartering
      • Dot-voting
      • Story mapping
      • Card sorting
    • Ideation clearinghouse
      • Capturing the imagined final product.
      • Complete in an hour or less
      • There is a tremendous cost to not knowing what’s in the head of your project’s stakeholders (i.e. cutting off the project because you didn’t “get the vision”)
      • Step 1: Set focus/boundaries for the workshop
      • Step 2: Do a warm up; introduce people to the raw materials
        • 4-5 minute time window
        • Write every feature you expect on a separate sticky
        • Becomes a feature palette for sketching
      • Step 3: Sketching timebox
        • Give 5 minutes
        • Ensure safety; it’s okay if you “can’t draw”
        • Everyone in the room sketches SOMETHING
        • Sketch individually
        • No rules — someone wants to write text, they can
        • Clarify that this is research, NOT design.
      • Step 4: Critique
        • 2-min round-robin where people explain the vision/design
        • Take careful notes, attach to respective sketches
        • Look for and work to resolve differences in vision among stakeholders
    • Designing with workshops
      • Learn, apply and recombine workshop patterns
      • Cardstorming + dot-voting + sketching, etc.
    • Pairing
      • an intensive one-on-one passing game
      • Continuous problem debugging and knowledge distribution
      • One person “drives” and the other “navigates”
      • Use to debug and fix code in a continuous and collaborative way.
      • Change partners each day
    • Cross-functional pairing
      • Designing in multiple dimensions at the same time
      • Better collaboration means less/lighter documentation
      • Move from whiteboarding/sketches straight to building things together.
      • Developers can help with the transition from talking about building to actually building
      • Each medium/perspective informs the other
      • Move from wireframes to building the structure to HTML/CSS in a simultaneous process.

Todd Zaki Warfel: The Design Studio Method

  • Borne out of architecture and Industrial design
  • Three phases (charette):
    • Create: generate and prototype the concept
    • Pitch: presenting the design concept
    • Critique: talking about the design concept
  • Approach: 6.8.5
  • Tools:
    • Butcher paper
    • Sharpies (rougher feel; it’s about generating ideas)
    • Personas
    • Design challenge/scenarios
    • Templates, e.g. 6-up or 1-up grids
    • STOPWATCH
  • Teams should be CROSS-FUNCTIONAL. Represent 3 viewpoints:
    • Design side
    • Business side
    • Development side
  • Each team gets a different persona, and ONLY focuses on their persona for the duration of the studio
  • Design studio kicks off the design process
  • Step 1: Create
    • Create 6-8 concept sketches individually, within 5 minutes
    • Give templates with 6-8 blocks (like a storyboard) with grids and space for notes
    • Either storyboard a flow, or use the time to sketch multiple approaches to the same screen
    • Don’t forget a warmup session
    • Use paper and pencil to get people out of their element/comfort zone
    • Don’t worry about details; just get the elements down
    • Goal: get just enough of the concept on the page to be able to pitch your idea to the team
    • Don’t create more than you can pitch in 3 minutes
  • Step 2: Pitch
    • Put all sketches on the wall with personas, “inspiration gallery”
    • Everyone gets 3 minutes to pitch their idea:
      • What did you intend to achieve?
      • How does your idea achieve that goal?
  • Step 3: Critique
    • While someone’s pitching their concept, nobody is allowed to speak.
    • Once the 3 minutes is over, team gets 2 minutes to critique the concepts.
    • This is NOT feedback, i.e. “I don’t like this” or “I like this.”
    • Critique based on the goals for the design: where it achieved or failed to achieve the goals.
      • 2-3 ways it achieves the goals, EVEN IF YOU HATE IT (green markers)
      • 1-2 ways it can be improved upon, EVEN IF YOU LOVE IT (red markers)
  • Step 4: take feedback and iterate on designs
    • Stealing is highly encouraged; if another team does something you love, steal it and make it better.
  • Benefits
    • Generate two weeks of work in less than six hours (420 different design concepts)
    • Instant buy-in from team members: since team members help create the designs, they’re already buying into the concepts
    • Get team members to own and invest in the concepts
    • Engineers said their estimates on time were 50% more accurate
    • Having five minutes as a rule means you don’t have time to worry about it; just execute
    • Everyone gets to vet concepts as they go: it’s not as good as usability testing, but the benefit is that only the strongest concepts survive

Jonathan Berger, Code Literacy for Lean Teams

  • Coding as literacy
    • gets rid of the binary nature of “you can code or you can’t”
    • If you work in technology, the medium is power
    • “I can get this code to work or not” can be frustrating to those just learning
  • LITERACY DOES NOT EQUAL FLUENCY
    • Just because you can code doesn’t mean you’ll be good at it
    • Getting technical doesn’t have to be a full time commitment
    • Many resources available (books, sites, tutorials, etc.) are geared towards people who want to be full timers
  • More and more shops won’t hire designers who can’t code.
  • How does code literacy help lean teams?
    • More literacy leads to more sustainable projects and more successful outcomes
    • Fine details are important, but often get shoved to the wayside by devs moving on quickly to the next feature
    • There’s a whole class of things that are easier/less time consuming to do than to explain
    • Being able to fix details yourself saves the developers time,
    • Gives team more hands on deck, fewer bottlenecks
    • Builds camaraderie
  • How does it hurt?
    • Time/cost in learning curve
    • Potential for overstepping bounds (humility is important)
    • Harder to look at things with an “innocent” eye (i.e. without thinking about how it can be made)
    • Tunnel vision in devs can skew priorities towards problems that are “easy” to fix
    • DON’T LET CODE OVERSHADOW DESIGN
  • Ways to become literate
    • Use in browser mockups (HTML/CSS/JQuery)
    • Use Inspector (Chrome) or Firebug (Firefox) to inspect and change elements, then import changes into code mockups
    • Align product/development/design
    • Set up a local dev environment (e.g. MAMP, Rails)
    • Get started with Git
      • Designers can contribute small changes, e.g. copy or CSS, and push to production without interrupting the developers’ workflow
      • You can make the fixes that matter to you but aren’t crucial to the developers
      • Watch out for breaking tests (e.g. when copy is changed)
    • Learn JQuery & the DOM
      • Make better mockups—directly in the browser
      • Make more changes for the team
    • Learn the language your team is developing in—PHP, Rails, Drupal, etc.

Session Notes, Sessions 4–6: Agile UX Summit, 2/25/12

Tomer Sharon: Getting out of the (Agile) Building

  • How do you do UX Research in an Agile environment?
    • OLD: Fast Usability testing rounds
  • Tips for Agile Usability testing
    • Make it relevant to what the team is doing NOW.
    • It should be a small thing, not a BIG thing, and do it 3-4 times per sprint (i.e. once a week)
    • Recruit participants in advance.
    • Plan really fast.
    • One-page usability test plan on Smashing Mag: http://uxdesign.smashingmagazine.com/2012/01/26/ux-research-plan-stakeholders-love/
    • Make it a team thing. If nobody was there to hear about it, it didn’t happen.
    • No reports. Try a spreadsheet with observations, and just color or code where observations repeat.
  • “Getting out of the Building”
    • Getting an understanding of what the users actually need;
    • What you find isn’t necessarily going to relate to what the team is doing right now.
    • Bring users in in some way.
    • “Virtual Visit”
      • schedule phone call with potential or existing user of product.
      • No specific goal other than having engineers, project managers, etc. to “see” a user.
      • Use screen sharing software to watch customers using the product.
      • Get the product team talking to the user, asking the questions.
      • If the same problem is noted multiple times, it may turn into a product change/sprint.
    • “Field Fridays”
      • Take some engineers to visit a client for 60-90 minutes
    • “Collaging”
      • Ask participants to use collage to express their feelings about a topic.
      • Collect images around the topic.
      • Gives people a chance to express themselves more easily around the topic; uncover user needs in a less direct way.
  • Getting buy-in for user research
    • Become one with the team. There is ONE team who works on creating something. If you understand that, you can do these things.
    • If we want to get buy-in, our route to work should include stopping and talking with people within the organization
      • Identify what people care about.
      • Identify new research questions.
    • It’s OUR research: Getting buy-in for UX Research projects.
    • Elsevierdirect.com Code MKFLYER for 20% discount.

Andres Glusman: Managing your Malkovich Bias

  • Building something isn’t the problem; getting people to *use* what you build is.
  • On track to do 600-800 tests a year
    • Test 2-3 days a week
    • Setup tests prior to knowing what will be tested
    • Objectives are discovered via conversations, etc.
    • Recruit participants from community and through recruiter (compensate for their time)
    • Use GoToMeeting to broadcast sessions to developers
    • Record sessions using Silverback, JUST IN CASE you want to view it again
    • Follow up each session with discussion and notes/video
  • Everyone suffers from Malkovich bias: the tendency to believe that everyone uses technology like you do.
  • Watching people use the stuff you build helps solve the Malkovich bias
  • Experiment then double down;
    • First question: what are people going to respond to?
    • Test the most simple concepts you possibly can.
    • Use A/B testing to test experiments.
    • Know that your most likely outcome is failure; look for the points of heat (where people are staying)
  • Look at short and long-term experiments
    • Short term: trying out new messaging or graphics
    • Long term: creating new service areas, profiles, etc.
  • Lessons learned:
    • Give team direct access to customers
    • Look for the boulders in the road; be comfortable with error
    • Substitute frequency for precision; if one screws up, there’s another right behind it
    • Have discussions
    • Launch and Learn
    • Accept that failure is the most likely outcome
    • Experiment, then double down. Once you’ve found something that works, start working on implementation.
  • Other lessons learned:
    • Don’t buy your coworkers books and expect them to read them
    • Don’t try to sell the process; people don’t want to buy the process, they buy the results
    • JUST START DOING IT and let the results show you how successful it is.
    • Snuggle up to failure
      • We are more often wrong than we are right.
      • What would you do if you assumed you would fail?
      • Go after what will cause you to fail as often (and as FAST) as you possibly can.
      • The longer you put off getting people in front of your work, the longer you can live under the delusion that you won’t fail.

Lane Halley: Quick, Visual, Collaborative and Continuous

  • The old days of waterfall: PLAN-BUILD-TEST-SHIP
    • Based on shipping physical discs that the user would have to install.
  • Then, we started having:
    • Web services
    • Web apps
    • Web analytics
    • Mobile devices
    • “Permanent beta”
  • Things don’t have to be built in one big chunk; you can iterate and learn as you go.
  • Many UX processes are still stuck in the Waterfall “over the wall” mode
  • Changing role of design
    • Design as team responsibility, not a single designer’s responsibility
    • Cross-functional pairing
    • Designer as facilitator, not a lone wolf
    • Team as product owner
    • Bringing things into the entire process: smaller, lighter, more continuous
  • UX finally has a “seat at the table;” but the processes we’re using are causing backlog and distress
  • How to make it “leaner?”
  • Team discussions: who do we want to talk to, and what do we want to learn?
    • Create an interview guide with themes
      • Intro:
      • About you:
      • Collect a story about what you want to talk about
      • What devices do you own, and what do you use it for?
  • Provisional personas
    • Defines user hypothesis
    • Shared visual artifact
    • Evolves over time
  • Get “permission” to interview users: start by asking the team who the users are. What are their pain points? How do we delight them? If the team can’t answer, that’s a step towards getting permission
  • Interviews can give you an insight into personas (i.e. USERS) you didn’t know existed.
  • Team Interviewing
    • Pair interviewing
    • Multiple note-takers
    • Group synthesis
    • Visual radiators, sketchboards
  • Continuous customer engagement
    • Five users every Friday
    • “Talk to us” or “give us feedback” button on the website
    • Just-in-time recruiters
    • The point isn’t how many people you talk to; it’s that you’re doing it regularly
    • Schedule time in advance, EVEN IF YOU DON’T KNOW what you’re going to talk about
  • Use one session for multiple activities
    • Listen, gather context
    • use collaborative activities
    • get product or prototype feedback
  • Collaborative design sessions
    • Put everything on the board
      • “You can only have 5; which ones?” then prioritize.
      • “You can have 5 more; which ones?” etc.
    • Collect everything and start talking about priorities, hierarchy, etc.

Session Notes, Sessions 1–3: Agile UX Summit, 2/25/12

Eric Burd: Want to win with Agile? Pivot your culture.

  • If you don’t have everyone on board, you have a problem.
  • How do you get the business side on board with lean/agile?
  • How do you set expectations on Minimum Viable Product?
  • Focus on one clear end goal, and try different ways of getting to that goal; “Keep one foot on Everest”
  • Two groups to win over:
    • Dev Team: they’re the ones who “get it.”
    • Sales, Marketing, Ops people
  • Many articles, books, etc. focus on winning over the tech and product teams; who’s talking about winning over the rest of the business team?
  • Agile isn’t as much about mitigating the tech risk; it’s about mitigating the market risk—building something nobody wants.
  • The business side believes that WATERFALL is about risk mitigation
    • clear beginning, middle and end
    • “campaigns” and “budgets” — long term thinking
  • We can’t get these sides of the business (marketing, ops, finance) into an Agile process, but we can do better at informing them of what we’re up to.
  • Six tips:
    • Find or light a “fire:” tie it to another issue within the organization (e.g. morale of dev team)
    • Listen to the customers; by being user-centric in our thinking, we can build better products. HOWEVER, you can’t get execs to actually listen to focus groups or usability labs. What you can do is bring customers in to have “lunch” with the executives, and customers can help sell execs on the process.
    • Get sales team in on it. Hearing from sales (i.e. people who regularly talk to customers and are trained at knocking down objections) that the process will help the org can make great strides towards getting the rest of the team on board. If the sales team can start hearing that customers are having product-driven issues that you can fix with Agile/Lean, you’re halfway there.
    • Words matter. When we’re using jargon, being self-referential, we’re essentially talking to ourselves. Other folks don’t get it. Start learning the language of the people you’re trying to sell to.
    • Train the exec team.
    • Focus on small wins. Find small wins within the org, and attach those to agile/lean vs. focusing on a “big bang.”

Phin Barnes: Investing in Design

  • The building blocks of the web are getting bigger and bigger, and easier to manipulate by clumsier and clumsier hands.
    • modern languages (PHP, Ruby, etc)
    • cloud services
    • ability to code and test relatively quickly
  • The web has returned to the basics of customer development; design must follow. Four stages:
    • Problem Definition
    • Sketch or Prototype
    • Improve Solutions
    • Implement Best Solution
  • Mindset > Skill set
  • Rules
    • No Unicorns and No A**holes: avoid the person who “knows everything” and doesn’t listen
      • Someone who listens and pivots improves over time
      • Someone who develops a “vision” and argues for it won’t listen to feedback; won’t listen to data.
      • Avoid making multiple minor iterations on the same bad vision.
    • Look at the feedback loop process
      • It’s easy to fake some of these things, i.e. talk the talk
      • If there’s one place where craft really counts, it’s in the feedback process
      • avoid comments that don’t move the project forward, or aren’t actionable
      • Feedback = an effort to understand where the person who created the feedback was coming from
      • Great feedback sessions are:
        • Casual
        • Scheduled
        • Planned out, in terms of what you want from the session
        • Organized in terms of decision points, and problem statements: what are you expecting to solve? What are you expecting the user to do here?
        • Watch people using the product, and videotape them; discuss it with the product team
    • What moves the crowd at your company?
      • Data, or a “visionary unicorn?”
      • At the best orgs, qualitative and quantitative data work together.
      • What are you measuring within your company?
    • If you’re going to adopt a design mindset, it has to be full staff
      • Get the rest of the org on board.
      • Make it tangible; find ways to get folks into the process and want to contribute to the product.

Josh Seiden: Replacing Requirements with Hypothesis

  • @jseiden; proof-nyc.com
  • Requirements and Hypotheses can both be used to frame the work of teams.
    • REQUIREMENTS: create an Internet Mouse that people can use when surfing the internet on their TV from their couch.
    • HYPOTHESIS: I think people will pay for a device that makes it easier to surf the internet from their couch, and you need to figure out what that might be.
  • For most teams, HYPOTHESES are a more effective way to manage your workload.
    • It’s very rare that you’re working in a condition of certainty, hypotheses let you operate in uncertainty.
    • Requirements take the THINKING out of implementation; team has no visibility to user/market needs.
  • When you give a team a problem, you engage the team’s creativity.
  • But you also need to give the creativity some boundaries, to keep moving the team forward
  • When you are building to an established standard, use requirements; if you’re creating something NEW, use hypotheses.
  • Hypothesis = Answer posed as a question; sets up an assumption to be tested.
  • Every design decision is a hypothesis that the market will test for you. If you can reduce the time between the design decision and the market feedback, you can drastically reduce risk.
  • Format of a hypothesis
    • Statement of what you believe to be true: We believe that…
    • A statement about how you’ll validate the hypothesis: We’ll know we’re right when…
    • Example: We believe that people will pay for a device that makes it easier to surf the internet from their living room couches and we’ll know we’re right when people can use our mockups without trouble and when people offer to pay for what we’ve built.
  • Process
    • Identify assumptions
      • What assumptions do you have about your customers?
      • That if proven wrong, will cause you to fail?
      • Who are they?
      • How does this product fit into their work or life?
      • What problem does it solve?
      • When and how is it used?
    • Express as hypothesis
    • Test riskiest assumptions FIRST
      • What are the highest risk assumptions that we know the least about? Put those into a backlog.
    • Break them down into testable parts
    • Use MVP concept to test your hypothesis
      • What is the smallest thing we can create to test our hypothesis?
    • Get out of the building: watch people to get feedback.
    • Lather, Rinse, Repeat
  • Use story maps to get all your requirements on one wall