progress bar

A.I. powered coronary angioplasty system

Summary

I joined the Abbott Vascular Research & Development team (AV R&D) in 2019, shortly after the scope and product direction of Ultreon 1.0™ was scoped out. Working in tandem with the senior designer, I contributed to the refinement of the UI (50+ screens and 20+ components and the component states used throughout the system), navigation, and overhauled 3 major workflows used in the software. In total, I sketched up, designed, and presented, over a 1000 design concepts to my manager and Abbott leadership.

After supporting Ultreon 1.0™, I was able to move-on to be the lead designer for major projects such as, the Satellite (touch screen devices), ThingWorx™ (data management) and finally, I led the design of the Ultreon 2.0™ software update from beginning to end. Ultreon 2.0™ incorporated the traditional guided workflow from Ultreon 1.0™, but also incorporated Flexible Mode and a 3D mode, augmenting a vessel in 3D space. The design team consisted of 2 designers, myself and my manager and 2 fulltime co-op designers who reported to me.

Accomplishments
  • Patent Co-Author.
  • CES Design 2021 Honoree
  • $380 million annual AV R&D sales (double the year prior)
  • Lead designer for three major programs. Received feedback and advice from my manager, the AV R&D senior designer, during meetings reviewing my work and objectives.
Responsibilities
  • User Research (hosted 20+ interviews with physicians and synthesize data)
  • Traveled to a Catheter Lab in Wichita, Kansas to observe angioplasty surgeries using our technology
  • Refined UI for Ultreon 1.0™. Was lead designer for Satellite and Ultreon 2.0™ software updates.
  • Hosted concept design review meetings with stakeholders (Marketing, HFE, Clinical, R&D Leadership, Director, etc).
  • Tracked designs as they were pushed through the design approval pipeline
  • Incorporated newest design techniques, designed for the 21st century user, while balancing user-centric designs, with input from 4 different R&D disciplines/teams, such as Quality, HFE, Marketing, Clinical and Engineering.
  • Collaborated with engineers to create a UI Architecture for developers for 4 major screen templates used throughout the system.
  • Designed and documented new UI components and states that could accommodate multiple input types (mouse/keyboard, touch & joystick inputs)
  • Designed UI for navigating and augmenting vessels in 3D spaces
  • Designed and documented a series of transitions and effects that would be used in the software
  • Worked with our legal team to ensure the design teams’ patents were drafted and filed correctly.
  • Supported developers with everyday questions around functionality and implementation. Elevated edge cases to senior engineers and project managers, where the workload of the 20 front-end and back-end engineers would be balanced using the Agile methodology.
  • Interviewed, onboarded, and managed two co-op designers, assigning tasks to help support the engineers’ work, while giving the designers freedom to explore new concepts, this in addition to carrying out everyday duties.
...

The Product

Streamlined and intuitive, Ultreon™ 1.0 Software gives better insights to optimize patient outcomes through automation and an improved workflow.

Ultreon™ Software allows interventional cardiologists to see inside the heart during percutaneous coronary intervention (PCI) and act on the obtained insights.

Ultreon™ Software guides physicians through PCI step-by-step following MLD MAX workflow and provides insights on morphology, vessel sizing, stent placement and post-stent optimization for more accurate decision-making.

Ultreon™ Software is powered by artificial intelligence (AI) that enables automatic quantification of calcification and vessel sizing.

How do we retain experienced users while growing into new markets with inexperienced users?

Case Study

Problem

Experienced physicians were unwilling to train novice physicians or physicians from the global market on Ultreon™ 1.0 software. Although the Ultreon™ 1.0 software launch successfully addressed the needs of a more global audience and was widely adopted around the world, we discovered that experienced physicians were not fond of the step-by-step approach presented, they would not train aspiring and new physicians on the software, creating friction in real-life adoption of Ultreon™ 1.0.
Ultreon 1.0 (2020)

Key Drivers

...
Global Market

The global market represented a multi billion dollar oppurtunity, but has slightly different needs than those observed from traditional markets like Japan and the United States. Targeting the global market netted Abbott Labs $380 million annual sales in the first year of launch, up from roughly $180 million in prior years.

...
Diverging Approaches

The traditional markets (US, EU & Japan) are used to more complicated, all-in-one software solutions with little guidance.

Growing markets around the world had presumably less training with existing software and required more guidance throughout the procedure and were more likely to rely on A.I. to make decisions.

...
Gate Keepers

Physicians around the world rely on experienced physicians, who are often trained in traditional markets. Experienced physicians were unwilling to learn new software that took a step-by-step approach to the procedure, even though it was widely pursued by physicians in growing markets or by novice users.

User Feedback

Human Factors Engineers along with R&D Marketing personel worked with Abbott Global Markerting to set up interviews with 50 physicians around the world. The physicians were split into groups such as experienced and inexperienced, and then again into those from traditional markets (US, Japan and Europe) and those from growing markets (China and South East Asia). These interviews continued over a series of months and gave R&D staff, including the design team, unadulterated access to the catheter lab and the inner workings of the angioplasty surgery. These interviews allowed the Abbott team to collect information on stage 1 and stage 2 proposals, until we were confident with our winning forumla which was pushed immediately into production, under the name as Ultreon 2.0.
...
...
(Left) Aptivue Software in a catheter lab (Ultreon’s predecessor), (Middle) reviewing stage 1 design prototype with Dr. Rivera (2021) & (Right) a clearly staged photo of me mapping feedback (2019).

The Design Process

Stage 1 Concepts

Primary Information
...
Secondary Information
...
Tertiary Information
...

Stage 2 Concepts

...
...
...

Component Library

The Ultreon™ System was built to accommodate scalability in a way so that components can work well alongside each other, at the same time, render well on various device resolutions. Ultreon™ was built on the Universal Windows Platform (UWP) by Microsoft, allowing for customization on their templated component library.

...

Ultreon 2.0

In the Ultreon 2.0™ release (2022), the system supported both Guided Mode, the main workflow available in the original Ultreon 1.0™ software, as well as Flexible Mode. Flexible Mode does not walk the user through each individual step of a procedure like Guided Mode, but is a one-stop-shop for all the physician’s possible needs and provides the tools they may need to use. Guided Mode is aligned with the needs of novice and infrequent users, walking them through each of the four major decisions needed during a procedure. Flexible Mode is aligned with the needs of the pros and the researchers who have sufficient experience in carrying out and reviewing notable cases that require Optical Coherence Tomography (OCT) software, like Ultreon™. Ultreon 2.0™ capitalizes on the contextual menu functionality, allowing for intra-panel functionality like, zoom/panning, measuring, editing, adjusting computation zones, access to a 3D augmented view of the artery, etc.

...

“(Ultreon 2.0) addresses the problems we are currently facing with Ultreon 1.0”

- Dr. E. Schlofmitz

...
...
...
...
...
...
...
...
...
...
...
...

Design Approvals

The Design Approval process was something my manager and I helped establish at the company. It allows for physicians, lead engineers and leadership to review design concepts from the moment of inception through to the implementation stage. This allows for frequent check-ins at important milestones for every design concept. Every element of the design and the intended functionality is reviewed for feasibility and their alignment to the greater Abbott Vascular strategy is assessed. Below is an example of a bird’s eye view of designed functionality for the Ultreon 2.0™ program as they were tracked through the design approval process. Since I was the lead designer for Ultreon 2.0™, I actively designed, tracked and planned review checkpoints for up to 4 different major functionalities simultaneously. Coordinating with a wide range of teams, such as the A.I. Team, Product Owners (lead engineers), the Global Marketing team, the Human Factors Engineering team, client-side and back-end engineers as well as helping the Quality team refine and push out requirements and test plans.

...

Implementation

Once designs are approved the design team moves to implementation phase. In this stage, the design team will not only ensure that all designs shared with developers are accurate and are up-to-date, but will also create a series of guides. Guides could be focused on the following: Dynamic Resizing, how the UI should adjust for different screen dimensions. Interactions, how the user can interact with the software using different input types. Transition Animations, custom animations using Microsoft UWP frameworks. The goal of the implementation phase is to get the developers’ work to look and function as close as possible to the finalized designs. In the implementation stage, the design team would also help uncover and solve bugs, or edge cases discovered while working with engineers and other quality teams.

Architecture: Dynamic Resizing

One of the requirements for the Satellite product was that the UI would have to automatically detect it and accommodate for the fact that it was being displayed on a touch device / monitor. The touch device had screen dimensions of 1920 x 1080 while the traditional screens had dimensions of 1280 x 1024. This required a design grid to be created to demonstrate how different screen templates would adjust to their new environment. Some elements on the screen would be fixed in size and only adjust position dynamically, while other elements and spaces would change size all together, all of which was defined for developers.

...
...

Interaction Guide

The purpose of the Interaction Guide is to show developers how and where a focus state would move around a screen. The guide also details how the different buttons on the joystick (table side controller) control different elements of the screen and navigation.

...
...
...

Transition Animation

Below are three examples from the animation guide showing developers how the on-screen transitions and effects should work. The documentation was based on Microsoft Windows Universal Platform (UWP). Other animations provided to developers or shared with physicians and HFE staff, were fully interactive prototypes.

...
Legend ... Type 1 (Drill) ... Type 2 (Fly out) ... Type 3 (Refresh)
Type 1 (Drill) ...
...
Progressing:
  • Screen transition initiates immediately after component (Room manager) selection animation is completed
  • Origin screen components scale to 150% over 0.20 seconds
  • Origin screen components' opacity is reduced to 0% over 0.09 seconds with a 0.11 second delay
  • Target screen components scale to 100% from 30% over 0.20 seconds after a 0.04 second delay
  • Target screen components' opacity is increased from 0% to 100% over a 0.09 seconds and after a 0.05 second delay
Digressing:
  • Screen transition initiates immediately after component (back button) selection animation is completed
  • Target screen components scale to 50% from 100% over 0.20 seconds
  • Target screen components' opacity is reduced to 0% after 0.09 seconds after a 0.10 second delay
  • Origin screen components scale to 100% from 150% over 0.20 seconds after a 0.12 second delay
  • Origin screen components's opacity is increased to 100% from 0% for 0.09 seconds after a 0.13 second delay
Type 2 (Flyout) ...
...
Progressing:
  • Screen transition initiates immediately after component (mega menu) selection animation is completed
  • Origin screen components slide left off screen for 0.20 seconds
  • Origin screen components' opacity is reduced to 0% over 0.09 seconds with a 0.11 second delay
  • Target screen components slide right onto the screen for 0.20 seconds after a 0.12 second delay
  • Target screen components' opacity is increased from 0% to 100% over a 0.09 seconds and after a 0.14 second delay
Digressing:
  • Screen transition initiates immediately after component (back button) selection animation is completed
  • Target screen components slide left off screen for 0.20 seconds
  • Target screen components' opacity is reduced to 0% over 0.09 seconds with a 0.11 second delay
  • Origin screen components slide right onto the screen for 0.20 seconds after a 0.12 second delay
  • Origin screen components' opacity is increased from 0% to 100% over a 0.09 seconds and after a 0.14 second delay
Type 3 (Refresh) ...
...
  • Screen animation initiates immediately after the component (add button) selection animation is completed
  • Target components (right side) move up 500px for 0.38 seconds after 0.16 seconds after the component (add button) is pressed
  • Target screen components' opacity is increased to 100% over 0.26 seconds immediately after component (add button) animation is completed.
  • Origin components slide down for 0 seconds after a 0.02 second delay (add button is pressed)
  • Origin components' opacity is decreased from 100% to 0% over a 0.00, immediately after component (add button) animation is completed.

Workflow Analysis & Consolidation

Settings screens follow a templated example. Joined the team late, was a previously used workflow across settings. Captured several examples of workflows being used, probably implemented by different engineers. consolidated on the best workflow —> approved by project owner.

...
...
...

Next Project:
Software management & analytics

Abbott Laboratories
2019 - Present
See more

John Lloyd

UI Designer
Boston, MA
john.s.f.lloyd@gmail.com