With Bonnie Evangelista, US Department of Defense
Key Takeaways
A significant portion of inefficiency in government contracting can be attributed to the writing & processing of long-form documents. Internal buyers from the four branches must write these documents to describe their needs and vendors must write them to respond. One forward-thinking group has already found promise in resolving this inefficiency with LLMs.
Topics Covered
- Background on Bonnie's role at the DoD I work for the Chief Digital and Artificial Intelligence Office at the DoD, which reports directly to the deputy secretary of defense, and sits on top of all the branches (Army, Navy, etc.)
- The mission of the CDAO is to find ways to "digitally transform" the DoD, and specifically how does the department get capabilities into the hands of soldiers as fast as possible
- Bonnie has been doing federal contracting her entire government career
- If someone wants to buy e.g. a tank, how exactly does that happen?
- First step is defining requirements, so if you want a tank, how fast does it need to go, does it need to be ruggedized, etc.? Someone in Bonnie's department helps with that requirement definition
- Once you have a requirement, then you need to figure out how to acquire that capability; that's the role of the contracting group, which manages the regulation associated with how we spend taxpayer dollars, ensuring fair competition, etc.
- Then there is source selection, so once proposals are received, there's a panel of stakeholders that determines who can best deliver the capability, or solution addressing the requirement; source selection itself can take months
- Everything above can take 12 to 18 months for a single contract, and that's what we're trying to address when we talk about emerging technology
- Where do you see those opportunities to accelerate the process? Where do you see opportunity for new technology?
- "We need to stop definining requirements and and start defining problems"; government needs to stop trying to predict what's going to happen in industry; it's a matter of asking for a faster mode of transport, so you can get cars instead of saying you want faster horses
- How do you envision this changing?
- It's more baked into policy than law; not everything we do needs to be done
- What are those technology projects you're working on to help accelerate this process?
- The Solutions Marketplace is one of the ways we're trying to do things differently; industry can state what solutions they have and what problems they solve, all in a 5 minute video like in a VC pitch
- All videos are accessed by a peer panel; if you're determined to have technical merit, you are then "awardable", which means a government buyer then has a mechanism to purchase directly from you
- Effectively you've already satisfied the competition process, so we can skip that whole process, which can accelerate the contracting process tremendously
- You can use the marketplace not just for contracting, but also market research -- buyers can figure out what's available much more easily
- Ultimately this is meant to be an easy entry point for both buyers on govt side as well as on vendor side; enables buyers to engage in much more collaborative manner, and vendors to not need to go through such a burdensome government contracting process
- This is not meant to replace e.g. office supplies and janitorial services; it's meant to be bleeding-edge technology that you want to buy now because it might be obsolete in 3 months
- Let's now talk about how you're integrating LLMs in the contracting process
- We wanted to experiment in this space, so I issued an open challenge to industry, willing to pay some money for any tool that could help levergaing OpenAI's technology
- We got one response, someone saying they could help with proposals
- In the MVP which took 45 days to build, I saw what generative AI could do; this was 9-10 months before ChatGPT came out
- The MVP could help customers (buyers in the DoD) to help define their problems -- that can be very hard if you've never done it before, and it's where we start the conversation with all of our customers -- it's the intake process
- I kid you not... what would usually take weeks or months I can now do in 30 minutes
- I can now start in plain language with the customer and translate that into something more specific
- At first the bot will generate a couple sentences, and then you can expand and expand with human feedback
- Then you have 85%-90% of a problem statement that I can give to industry to see if they have solutions
- What's on the roadmap is the AI starting to recommend solutions from the marketplace as you're writing out the problem statement
- Due to the risks around AI, real or perceived, government will be longer to adopt
- Though in certain areas, it may be that you need to compromise on the risk of the AI managing something because it is just that much better than a human; if you're closer to the front lines of battle, that may matter more
- What advice do you give entrepreneurs about how to best engage with you?
- Get clear on your value proposition, and how it helps end users (ultimately the soldiers in most cases -- they are the bread and butter of the department, and if you're not helping them, we can't help you)
- Get on the solutions marketplace; if you're on there, we can now buy from you
- There is more work to do to improve it, but the marketplace is the easiest way for anyone to sell to the government
- There are some of us within the department who are trying to move the needle in the right direction; you just have to find them