Improve CSAT of
Contract Management Platform
Enterprise platform for Contract Management
Key highlights of this project
Overview
Role & Duration
Icertis is a privately owned software company that provides contract management software to enterprise businesses using a software-as-a-service model.
Icertis' contract management platform turns contracts from static documents into a strategic advantage, giving companies the powerful capabilities to become contract intelligence.
Client
User Experience Architect
Drive entire engagement from research to design in around 8 months with help of Senior UX Designer and other team members.
Cross Functioning Team
•VP and Architect of Engineering
•VP and Project Manager of Presales
•VP Product Management
•Professional Services (Business Analysts)
•Sales Support (Customization)
•Commercialization team (Implementation Team)
•Product Documentation Team
July 2018 to May 2020
Project Summary
Business Objective
Increase revenue by creating a delightful experience for the existing customer base.
Success Criteria
-
Increase in CSAT score for the identified workflows which has lower CSAT score
Use Goals
-
CreCollaborate and take carrier guidance from Allumini
Challenges
-
No access to users. Solution hypothesis created instead of user interviews.
-
Persona-specific navigation doesn’t work in role-based workflows
-
Prioritizing features with the product and engineering team for Search and Navigation
Outcome
Increase in CSAT score after testing the beta release of the application for the identified workflows.
-
Successful release of the application for alumini and students.
The Problem
•Google (Product Name: gContracts) – The customer of the organization has conducted an internal CSAT survey of using the platform and scored 27% in overall satisfaction.
•Brief for UX is to Increase the overall CSAT score for the product from 27% to 68%. (68% is the mean in the normal distribution curve where the major user population lies.
Product Discovery
I started with product discovery to understand available data with internal and external stakeholders which includes:
•Connect with Internal & external stakeholders about user request tickets for new features/customization
•With Business Analyst about new feature/customization requests
•Customer Support about service requests
•Evaluate customer backlog items
•Connect with Product Management about the product roadmap
User Inputs:
40% of the backlog for workflow used by the Legal team
Challenges: No access to users
Directions: Evangelize importance of user connect with stakeholders
Challenge: Understand user pain points
-
400+ service requests
-
1200+ Customer backlog (based on service requests and product enhancement)
-
Difficult to differentiate UX and technical challenges..
-
Performance (data loading) issues logged under UX issues
Action: Manual analysis of request data
Result: Not able to find correlation in backlog items
Unorganized Backlog Data
Not able to identify correlation in data
Challenge: No access to real users
Customers (stakeholders) on behalf of the “User” were providing inputs (tickets).
No access to the customer’s end user for interviews/contextual inquiry.
Not possible to implement an analytics tool due to confidentiality issues.
Reasons: Company policy not to send data outside (confidential data leak) and other obligations
Action: Evangelize importance of user access with customers to understand pain points. Identify & Validate Method to do user research
Result: Agreement on cocreating solution with real end users to solve their problem
Understand Competitor
Understand Competitor
-
Study of similar applications in contracting space
-
IBM Emptoris, SAP Ariba, Contract Management applications on Salesforce (Apttus, Spring CM, Zycus, Conga)
Understand Technology Trends in same space
-
Angular UI
-
Command line search
-
AI & ML
-
NLP
Understand Existing User Base
-
The primary user of this platform is Legal Team. Most of the client users are in their 50s.
-
(Based on data from the customer support team) Challenging to adopt technology changes.
-
Other users of the platform are from different backgrounds and in the range of 25-50 who are tech-savvy.
Challenges: Empathize, primary user group,
Action: Propose experience to adopt existing and first-time or new users as a possible approach. E.g. Salesforce Classic and Lightening UI
Design Direction: Design experience which will accommodate both user types
Result: Agreement on a proposal from UX subject to technical feasibility evaluation and POC
Challenge: Difficult to understand the competition
Difficult to access IBM and SAP demo instance
Action: Able to get access to the Salesforce demo instance
Result: Able to get insides only for Salesforce
Challenge: Limitation of the current technology stack
-
The current UI is made in jQuery 4
-
Inline UI code is written for each UI screen.
-
Elastic search is an old version (V2) that doesn't support AI & ML
-
Clients are using different versions of the platform. Need to upgrade them to upgrade with a new experience
Action: Evangelize and Advocate to adopt new technology to improve overall user experience
Result: Agreement on a proposal from UX subject to technical feasibility evaluation and POC
Research Survey
User Survey was conducted with provision of qualitative feedback for each questions.
Number of questions varied based on customer domain for relevance.
-
Survey Tool: Survey Monkey
-
Number of Questions: 15+
-
Number of Customers: 3
-
Total Number of Users: 180+
-
Rating Scale: 1 to 5 where 1 is Not satisfied and 5 is fully satisfied
Questionnaire
1. Navigate to the required sections such as Agreements, Contracts, etc.
2. Use search & filter to locate relevant agreements on the agreements list page
3. Create an agreement (own paper) and go through the process of approval, internal review, signing, and execute
4. Create an agreement (third-party paper) and go through the process of approval, internal review, signing, and execute
5. Request for creating contracts and creating a contract
6. Create agreement templates
7. Create and add clauses to the clause library
8. Redline during the negotiation process
9. Collaborate with other members of the team while working on an agreement
10. Track obligations and commitments
11. Reporting and analytics on the overall contract portfolio
12. Inclusion of price annexures & other commercial or Inco terms which may get decided in a parallel negotiation
13. Signing of consortium contracts
14. Please rate the overall usability of the product
15. Please rate the overall ease of learning the product
16. Please rate overall satisfaction of the product
% Avg. Satisfied Users
(4 & 5 out of 5 Rating)
27
26
52
61
47
53
67
51
83
60
55
68
87
31
28
21
% Satisfied Users
(Google gContracts)
27
33
62
67
57
53
61
61
73
50
NA
NA
78
NA
NA
27
Quantitative Analysis
Type of User Profile
Frequency of App Usage
Fig: Screenshots of surveys conducted using survey monkey and analysis done on spreadsheet
Challenge: Identify the target user persona
-
Multiple personas
-
The frequency of usage is different
-
Partial survey completion by some users
-
The average rating given by some users for all questions
-
Missed to include education and age groups
-
Action: Slice and dice survey result received from Survey Monkey (Survey Monkey doesn't give detailed results)
Result: Correlation between type of organization, persona and frequency of usage
User Inputs:
Search agreements
Navigating to the required document
CSAT 27% (27% by gContracts)
CSAT 26% (33% by gContracts)
User Inputs:
-
30% of the users are from the legal department
-
48% of the users are uses application once in month and more
-
40% of users are 50+ age
Design Directions: Create an experience for non-frequent users
Qualitative Analysis
Qualitative feedback helped to identify user pain points for the workflow/feature which the user has given a low rating.
-
Search agreements (Where to search?)
-
Different search experiences on key pages
Fig: Screenshots of existing application
User Comments (Based on qualitative data from research survey)
“Multiple search fields are confusing”
“Search function isn't intuitive, and often I feel like if I forget one thing, I don't get what I need and end up having to click through 80 contracts to find the needle in a haystack. I wish it was more of a search engine/Google-esque function where you just typed in a keyword or name of contractor and saw only things related to that. I can't seem to make that work, if that function exists.
User pain points: Findability – Different ways of finding information
Design Direction: Propose a simplified way to find the information
-
Search agreements (What to search?)
-
OCR, Metadata, Both
User Comments (Based on qualitative data from research survey)
“It is super hard to find the relevant agreement I am looking for”
“Search functions need to be more robust and intuitive”
User pain points: Findability- What to search for?- Different data types/categorized to find information
Design Direction: Understand and help the user with what information he is looking for
Fig: Screenshots of existing application
-
Search agreements (How to search?)
-
Metadata filters, saved search, favorite, attribute selection
User Comments (Based on qualitative data from research survey)
“Saved search is not often used items. It is not intuitive”
“Left hand search is helpful but confusing. How to use d-7”
User pain points: Findability: How to search? – Different methods to find the right information
Design Direction: Provide help to reach the right information that user is looking for
Fig: Screenshots of existing application
-
How to Navigate
-
More time to reach to the desired page (Speed in usability goals)
User Comments (Based on qualitative data from research survey)
“Difficult to navigate and get to the document needed”
“Navigation to view actual supporting documents in the association is not intuitive and takes several clicks and each page take a long time to load”
User pain points: Guidance: How to Navigate?
Design direction: Propose simplified and quick ways to guide user for required information
Fig: Screenshots of existing application
Design Process
Personas and User Journey Maps
*This is reference persona of primary user. All other persona and user journeys were created as part of process and validated with product manager
Tina’s pain points:
“Multiple search fields are confusing”
“Search functions need to be more robust and intuitive”
“Saved search is not often used item. It is not intuitive”
“Left-hand search is helpful but confusing. How to use d-7”
“It is super hard to find the relevant agreement I am looking for”
“Difficult to navigate and get to the document needed”
•“Navigation to view actual supporting documents in the association is not intuitive and takes several clicks and each page takes a long time to load”
•Design Directions: Provide a simplified experience to “Find” and “Guide” Tina to reach the required information.
Solution Hypothesis
Proposed Solution
-
Provide simplified experience to Tina “Find and “Guide” (Search and Navigate) required information
-
Desired experience is expected to accommodate her as well as new members and non-frequent users
Solution Brainstorming
Fig: Screenshots of digital whiteboarding
Proposed Solution
Fig: Screenshots of existing application and proposed prototype
Goals Achieved
• Simplified search experience (Single search) across the application for Tina.
• Provide infographic and actionable content to provide more insides into her documents.
Test Assumptions and Hypothesis
Design Sprint Workshop
-
Design sprint workshop conducted to validate Tina’s Pain Points
-
Co-create design solutions with legal team members representing Tina
Workshop Details
Location: New York (Google Office)
Duration: 2 Days
Participants:
-
UX
-
Product Manager
-
VP Engineering
-
Engineering Team Member
-
End users (5)
-
Product Owner (Client side)
Outcome
Solution sketches validated with legal team members (users)
Fig: photographs of workshop conducted with users
Solution Sketches and Prototype
-
A personalized experience for Tina
-
Simplified search experience (Single search) to “Find” the documents for Tina
-
Provide a simplified way to “Guide” (Navigate) Tina to locate necessary documents and actions her role demand
Fig: Screenshots of digital whiteboarding
Fig: Screenshots of proposed prototype from workshop
Design Journey and Final Outcome
-
Personalized menu bar for Tina
-
Simplified search experience (Single search) to “Find” the documents for Tina
-
Provide a simplified way to “Guide” (Navigate) Tina to locate necessary documents and actions her role demands
-
Provide a simplified way to “Guide” (Navigate) Tina to the necessary documents she is referring by:
1. Horizontal navigation
2. Actionable widgets to locate necessary documents and actions her role demands
The Solution
Proposed Solution Based on Hypothesis
Accomplishments
CSAT Survey after design changes
Second User Survey was conducted with Google gContracts to understand satisfied users after solving Tina’s pain points.
Provide simplified way to “Guide” (Navigate) Tina to necessary documents she is referring by:
1. Horizontal navigation
2. Actionable widgets to locate necessary documents and actions her role demands
Survey Details
Survey Tool: Survey Monkey
Number of Questions: 15+
Total Number of Users: 67
Rating Scale: 1 to 5 where 1 is Not satisfied and 5 is fully satisfied
Overall CSAT for Google gContracts after survey: 43%
Workflows with new CSAT
Search agreements
Navigating to the required documents
CSAT 35% (previous score 27%)
CSAT 62% (previous Score 33%)
Fig: Screenshots of survey results from survey monkey and further analysis using Spreadsheet
Takeaways
Takeaways
Opportunities
-
Research & Analysis is a data-driven design technique. Identifying the right data is important. (What)
-
Empathizing Tina (user interviews) and co-creating design is important to validate data. (Why and How?)
-
Simplified search experience for Tina and new/non-frequent user which will help to:
-
Search agreements by use of simple English as these users are not tech savvy (cannot use query-based language)
-
Personalized results based on search history, recently used document, suggest document by type of contract she uses.
-
Platform to support other features for Tina and new users from her team. (e.g. Salesforce classic and lightning)
Retrospective
-
User interviews and Design sprints could have been done early stage to understand Tina’s pain points as surveys are a time-consuming method that may give random data.
-
Implementation (prioritization)of design solution for search could have been done to improve CSAT score
-
Usability testing of solution hypothesis mockups and design sprint mockups could have been done before deployment
Challenges, Learnings and Next Steps
Future Challenges
Learnings
-
Co-creation of design solution using design sprint by involving Tina
-
Technical architecture restructure
-
Upgrade Elastic search, Kendo UI framework
-
New product offerings such as NLP, Analytics, AI, ML
-
Learning curve and training sessions for existing customers
-
Development cost
-
Persona, age group & frequency of usage are driving factors
-
Design should speak the same language that the user understands (No query-based language)
-
Design should cater to new users as well as power user
-
Design should adopt new technologies and trends as the user already does
-
Hypothesis accuracy is directly proportional to the number of users you interact
Tradeoff
-
No access to users. Solution hypothesis created instead of user interviews.
-
Persona-specific navigation doesn’t work in role-based workflows
-
Prioritizing features with the product and engineering team for Search and Navigation
My Overall Contribution
Designer
Follow UX best practices to deliver delightful experiences to real users of the application
-
Qualitative Research & Quantitative Research
-
Prioritization of solution approach
-
Create a solution Hypothesis
-
Follow the design Sprint methodology
-
Create solution
-
Validate Design Sprint solution with Hypothesis
-
Build MVP
-
Test Hypothesis CSAT Survey
-
Understand the limitations of current technology
-
Understand the proposed technology
Evangualizer
Educate and Advocate UX best practices within (Internal Stakeholders) and outside organization (External Stakeholders)
-
Socialize Research Processes such as CSAT surveys
-
Socialize the importance of user interviews
Leader
Drive entire engagement by collaborating with stakeholders and another cross-functioning team
-
Coordinate with internal and external stakeholders to create CSAT surveys
-
Drive Design Sprint workshop with stakeholders
-
Drive overall project engagement from the UX side coordinating with multiple teams within an organization
More Projects
Smart Infrastrecture Management
Smart Infrastructure Management System