Questions to ask

State Software Budgeting Handbook

Questions to ask

When you are considering a budget request for a custom software project, it will be difficult to consult this entire handbook to find the right questions to ask. Here are some basic, open-ended questions that you can ask to determine if a project is set up for success.

What are the goals of the project? What outcomes are prioritized?

Wrong answer: Anything technical in nature, instead of about improving the user experience.

Right answer: One or more specific user needs are named.

What is the user need that this project will address?

Wrong answer: Anything that doesn’t name clear needs of end users identified via user research.

Right answer: The agency has determined specific needs based on interviews with end users, and can name several of those needs specifically.

If the selected vendor doesn’t perform adequately, how difficult will it be to terminate the contract? How long will it take to replace them with another vendor? How much do you think that will cost?

Wrong answer: "We would be very reluctant to terminate the contract. It would take months or years to replace them with a new vendor. Significant staff time would be required to do that, and it would set our project back by many months. Once we have a system, we’d have to start all over if we decide to change vendors."

Right answer: "It will be a time and materials contract, so we could stop assigning work to the vendor at any time, and that would be the functional end of the contract. We could reissue the RFP and have a new vendor onboarded within six weeks. It would require a small amount of staff time, and it would set the project back only by those six weeks."

Will the RFP include requirements for how the system will operate? If so, how many requirements are included?

Wrong answer: "We’ve spent the past year reviewing our business requirements, and we’ve written hundreds of requirements to include in the RFP, to ensure that we get exactly what we need."

Right answer: "We’re more focused on the outcomes we want from the new system. We’ve developed a backlog of user stories to help guide the team’s work, rather than producing a detailed list technical requirements."

How long do you expect the RFP will be?

Wrong answer: "We’ve developed several hundred pages of system requirements along with 50 more pages of standard terms and conditions."

Right answer: "Less than 20 pages, and we expect to keep this under the state’s simplified procurement threshold, to make it easier, cheaper, and faster for new vendors to bid on the project."

Do you anticipate issuing a fixed price contract, or a time and materials contract?

Wrong answer: "Fixed price, because it’s the best way to control vendor costs."

Right answer: "Time and materials, because it’s the best way to retain the flexibility we need to respond to user needs, manage to unforeseen technical challenges, and ensure vendors that aren’t delivering what we need can be changed without putting the project at risk."

What value will be delivered to the users within six months?

Wrong answer: "None — it won’t be ready by then. We plan to show it to users when everything is finished."

Right answer: Specific examples are named.

Who will be the product owner?

Wrong answer: "What is a ’product owner’?"

Right answer: A specific person is named, or they’re training in-house staff to take on this role.

What software development process will be used?

Wrong answer: "Waterfall," or any answer that indicates a lack of comprehension.

Right answer: "Agile," "Extreme Programming" (XP), or "Scrum" are all acceptable answers.

On the team that prepared this request, who has experience developing software?

Wrong answer: "Nobody."

Right answer: A specific person is named.

How often will work be deployed into production?

Wrong answer: "When it’s done."

Right answer: "At the end of each sprint."

Will the project automate testing? Integration? Deployment? Security tests?

Wrong answer: "We’re looking into that."

Right answer: "Yes, from day one."

How much will change orders cost?

Wrong answer: Any response that foresees change orders of any kind.

Right answer: "We expect the system to change constantly in response to new user needs, new technology and new policy so that’s why we’re using a time and materials contract and an agile development approach to lower the cost of responding to these changes."

How will you know if the project is on track and that contractors are delivering as promised?

Wrong answer: "We’re contracting with an independent verification and validation (IV&V) expert to provide us with monthly reports on the project’s status."

Right answer: "Vendors will provide frequent demonstrations of working software that reflect our priorities, meet the technical standards of the QASP, and provide value to end users. If these standards are not met, and value to end users isn’t shown within six months, they’ll be terminated."

Who will own the software?

Wrong answer: "The vendor."

Right answer: "The state" or "it will be committed to the public domain."