Our dismay about the pace at which the state is processing unemployment claims was compounded this morning after reading in the LA Times that the Administration blames an outdated computer system when we know that an outdated computer system need not be an obstacle. As a reminder, earlier today we sent legislators the notes we sent them earlier this month about Rhode Island’s conversion to a cloud-based system. Distribution of unemployment benefits is a critical state service the performance of which California should lead, not lag.
Dear Legislators,
See below for notes from today’s webinar about Rhode Island’s conversion to a cloud-based UI system. RI left its mainframe transaction processing server relatively untouched while using a new front-end to expand and automate its customer service and information collection processes. A copy of the slides is attached and officials interested in accessing the free codebase should reach out here.
Today’s webinar described Rhode Island’s modernization of its UI claims processing system to handle drastically increased volume due to CV 19. While much focus has been on old legacy systems, RI actually left its mainframe transaction processing server relatively untouched while using AWS services to expand and automate its customer service and information collection processes. For example, RI replaced its Interactive Voice Recognition (IVR) and Interactive Web Recognition (IWR) systems with Amazon Connect and Amazon Lex. This allowed the Department of Labor and Training (DLT) to handle a greater volume of calls when UI recipients have to recertify that they are still unemployed and report any earnings. This is separate from filing claims, and the strategy was to free up human call center capacity. The same strategy applied to using Amazon Lex to deploy an FAQ chatbot to answer common questions. As the legacy AS400 server remained in place to process and pay claims in nightly batches, DLT used AWS to deploy an API that allowed them to automatically verify income from Department of Revenue Data, as well as to preprocess PUA claims data submitted online. This would include checking for common errors and omissions and automatically emailing claimants for more information more quickly rather than waiting for the AS400 to choke on the error and deny the claim.
Phil Bertolini, Co-executive Director, Center for Digital Government
It’s hard to imagine just a few months ago we had an economy in serious growth mode. Now our economy has declined because of CV 19, and many people find themselves laid off. This dire impact has also been visited on governments ast they tried to keep providing services as a tidal wave of UI claims came on, a volume that was never predicted nor technically planned for.
Scott Jensen, Director, Rhode Island Department of Labor and Training (DLT)
Let’s explain the scale of these numbers. What sounds small for some states is enormous for RI. It’s a small state that’s densely packed. The record week for UI was 5300 claims during a banking crisis in 1992. CV 19 since March 9 has generated 12x that in one day. In the great recession, 130k people claimed UI in 2009, and in just the first 5 weeks of CV 19, there were 144k claims. RI’s entire workforce is 566,605 people. As of May 4, RI had hit 200k, 35% of the total workforce. Some of these may be duplicates, but that’s enormous volumes. In February 2020, we paid 2700 people. In April, we paid 72k claims. Every week you’re required to certify your information is accurate and that you continue to need UI. On the first Sunday of CV 19, 8179 people certified. Last Sunday we certified 90,486 people. Those numbers are staggering.
We put together this graph, and I’d like to point your attention to the claims coming in, where we assumed there would be a peak of claims that would then decline. If that happened, then we would quickly see that peak draw payments for a sustained period of time, which is why you can see the trust fund deficit grow rapidly. This is an educated guess as of March 18. Things came in a little slower than this, and it took us longer than we thought to process claims. I left the meeting showing this to the governor happy, since it seemed the governor had received the info she needed. I then began to think about needing to qualify eligibility and quantify benefits. After that, every one of those people is going to need to call a very old infrastructure here at the department to certify and would likely crash our telephone system. We knew there would also be a PUA via the CARES Act that would introduce others typically not eligible for UI into the system. You can’t code easily into a mainframe computer a whole new set of criteria to pay a whole new program while a giant wave is hitting you. Those two challenges kept me up at night. We figured out two ways to do it with the cloud. One was working with professor Hastings to build a new system for PUA and the other was working with AWS to build an IVR and IWR structure to allow people to certify their claims.
Stuart Venzke, State and Local Health and Human Services Executive, Amazon Web Services (AWS)
We’ve been inspired by the entire department of labor and their commitment to serving their citizens. One of our AWS leadership principles is customer obsession. We came across this tweet that showed RI leading the pack, but Scott termed that getting an “F+” since tens of thousands of claims were still outstanding. Our engagement began with the RIPL project, and then Scott laid out some of the technical challenges to paying claims quickly. They had a legacy payments system and an antiquated IVR and IWR system that couldn’t expand quickly. This situation was an almost perfect use case for the AWS cloud because it required elasticity to scale automatically on demand, agility to provision resources in minutes and a pay as you go model to buy only what they needed for as long as they needed. For example, PUA only runs to the end of the year. We quickly honed in on the IVR/IWR system that could be replaced with Amazon connect, our cloud based call center. We did this in 9 days. One design consideration was there couldn’t be real time integration with the legacy application, because it was old and complicated and because that team was full out working on other CARES Act required changes. We downloaded flat files with claimant info that we used as a lookup table to verify claims. We then implemented an FAQ chatbot using Amazon Lex, the conversational interface service from AWS. We maintained the DLT look and feel, and DLT put out explanatory videos and monitored social media for user feedback. Because folks have been certified the first thing Sunday morning, literally at 12:01 am, since people get paid earlier the earlier they certify, we didn’t have time for a trial run. We helped 75k people certify that first Sunday. We’re working to develop a backlog of user experience improvements.
Professor Justine Hastings, Director, Research Improving People’s Lives (RIPL)
We help governments set up research data lakes in AWS, which is often done by industry insight leaders like retailers, JP Morgan, etc. They optimize unwieldy data to improve the services they offer consumers. We ingest administrative data, anonymize it and make it accessible to analysts. The NSF Future of Work Project uses this approach to calculate ROI for job retraining programs. Workers don’t actually have info on the expected returns. A worker has to enroll in a program without a measure of how much value that program has added to employment and income improvements. Government partners own the data and software going forward that gives them capacity to pivot and expand on short notice. We had just demoed this software to Scott and other leaders, and he must have been anticipating the deluge of claims for PUA. He reached out and said I don’t think our systems will be able to handle this. Can this new system help? Having a good concept of how those programs work, I said yes and reached out to AWS who was a partner on the NSF grant. The point was to pull as much work as possible from the AS400 system. The new system built from the flat files exported from the legacy system allowed us to set up a new interface to collect data and then automate checking its accuracy and emailing users for clarification. As part of the future of work, we also were able to combine proprietary data with 3rd party data in a way that preserved copyright and confidentiality. This allowed us to validate income claims with Department of Revenue data, which is sensitive. We developed an API that ran a script between the two systems to access and confirm income without compromising anonymity or security. This runs automatically and saves time from having to do this manually. We packaged this entire step by step process and made it available for any state to adopt. We can connect anybody if this would be of interest.
Questions
How did you convince leadership to do this?
Scott
Never let a crisis go unexploited. We didn’t have a lot of choice when you looked at those numbers. I was literally driving home thinking what am I going to do? We had a real time version of the AWS reverse engineering process. I was thinking of what my poor press director was going to have to deal with when we couldn’t process these claims, so I called them at ten o’clock at night on March 25, and we were working on it the next morning. Then the discussion can be about if people are nervous about trying something different, what’s a better solution? If you don’t have one, what are your specific concerns? It did require an executive order to allow the tax department to share the data despite security, but we did get one within 24 hours. Necessity is the mother of invention, coupled with dealing with actual problems rather than abstract anxieties.
What made AWS connect the right solution?
Stuart
One was there were hard limits with the legacy IVR system, something like 72 simultaneous calls. Given 90k ppl certifying, especially over the phone, it was clear there would be a bandwidth issue there and frustration with busy signals. AWS can offer that, plus more reporting to provide better insight into the calls the agency is receiving. A simple example is the first day we went live, we looked and saw 150 simultaneous calls, and we asked how that compared to history, and the answer was we had no idea. We never knew what happened to calls beyond 72.
What’s the number 1 success factor in the collaboration?
Justine
I would say people. Leadership in RI embraced and were open to new ideas and new technology. The AWS folks are very customer centric and have a bias for action. They work quickly and have been a great partner on the future of work project. We have a group of people who value science, measuring impact and objectivity. Everyone was able to come together quickly to make this work. I really believe in specialization and gains from trade. We all have skills and these big challenges require different skills. When people bring their skills together, it’s amazing what we can accomplish.
What’s your measurement of success?
Scott
Our goal is to make sure that we’re helping people not only in enormous crises but in a day to day way into the future. UI is a tough experience. We’re all pinned to what we do for a living, which often shapes our identity. Our goal is to make the experience as, inspiring is the wrong word, but as respectful as possible. That means using tech to get claims out quicker, but the point is getting people through a rough time. We want them to be in better emotional shape to change their skills and improve themselves in a way they see fit. We want to make this experience as empowering as possible.
What’s your fiscal year?
Scott
July 1 — June 30
Do you have a separate group of workers monitoring social media?
Stuart
It was the DLT comms team that was monitoring social media.
How does AWS Connect triage calls? How are they responded to?
Scott
The calls we’re talking about are certification calls. We’re not talking about the call center, which is also a direction we’re going with Connect. This is an auto voice recognition system that says press 1 if you’re still employed, press 2 if you need to report wages earned, and there’s a webpage that does the same thing. It’s not as complicated as a 45 min person to person call. We were experiencing 900 calls/minute at one point.
What about auditing performance?
Scott
The PUA application essentially aggregates the information. It doesn’t bypass the regular system. It then feeds it into the UI system in batch at times that are convenient for the computer, early in the morning. Security protocols, etc remain the same. We’ll be working over time to get some of those functionalities off those mainframes. We didn’t code a whole new system, which is why we were able to do this quickly.
Justine
If you think about the AS400, there’s a lot of things they’re trying to do. It verifies info when a claim is submitted, and it will reject a claim that needs more information. The way to think about the AS400 is that it’s really good at keeping a record and sending a file to get paid. Everything else it does slowly. It’s not that hard to then replace those other tasks with another system. The goal is to feed the AS400 a really lovely file that makes it happy in one big batch.
Scott
We did this on the fly, so we didn’t automate as much as can happen. That remains to be done. What we did was prove that you can do what justine described.
