Picture by Ron Lach (Pexels)
Data science projects can be complex and challenging to manage, especially when there are numerous stakeholders involved. The success of these and other software projects often hinges on getting the initial requirements right. A structured approach can help ensure that the project goals are clear to all participants and the development team has a common understanding of all the requirements.
One way to achieve such common understanding is to follow the military standard 498 (or “MIL-STD-498”), which I found here a couple of years ago when I started in the industry. It provides a structured framework for the software development process, including requirements engineering. This standard can be used to create a software requirements specification (SRS) that documents the necessary information about the project and acts as a reference throughout the project’s development.
As data scientists, what we are shipping in the end is often a piece of software (some ml-model) which interfaces with databases or is deployed in a cloud environment. Therefore, the parallels to the software development process are quite obvious. During the initiation of a data science project, it is important to ask for the following points informally in a meeting:
- Capability Requirements: First and foremost, what are the project’s capabilities? What are the project goals? The team should clearly understand the desired outcome of the project.
- Characteristics of the Solution that are Conditions of Acceptance: What are the criteria that the solution must meet to be accepted? Identifying these early in the project can help avoid confusion and ensure that everyone is on the same page.
- Acceptance Tests: What are the tests that will be used to ensure that the project meets the criteria for acceptance?
- Qualification Methods: What methods will be used to demonstrate that the project meets the requirements? Will there be demonstrations or documentation?
- Summary of Advantages/Disadvantages: What are the pros and cons of the project? This information can help the team make informed decisions throughout the project.
- Priorities of Features: What features are most important? Identifying these early in the project can help ensure that the most critical features are developed first.
- Software Environment Requirements: What is the software environment in which the project will be developed? What platforms and tools will be used?
- Security and Privacy Requirements: What are the security and privacy requirements of the project? What measures will be put in place to ensure data security and privacy?
- Support Concept: What is the support concept for the project (or what should it be according to the client)? Who will (should) provide support for the project after it is completed?
- Interfaces: What interfaces will the project have? Will there be internal or external interfaces?
- Personnel-Related Requirements: What personnel requirements are there? Who will be responsible for various aspects of the project?
- Time Constraints: What are the project’s time constraints? What deadlines must be met?
- Training-Related Requirements: What training will be necessary for the development team and stakeholders?
By gathering this information and documenting it in an SRS, the development team can ensure that everyone has a common understanding of the project’s goals and requirements. This can help avoid misunderstandings and ensure that the project stays on track.
In addition, following MIL-STD-498 can help ensure that the project’s requirements are well-defined and that the team is following a structured development process. This can increase the likelihood of a successful outcome.
Once you collected all the requirements and achieved an alignment, there are a couple of points that you must follow up on:
- Requirements Analysis: Once the requirements are gathered, they need to be analyzed to determine their feasibility and impact on the project. This includes identifying potential conflicts and dependencies among requirements, assessing the cost and effort required to implement them, and prioritizing them based on their importance.
- Traceability: It’s crucial to maintain traceability between the requirements and other project artifacts, such as design documents, test cases, and code. This helps ensure that the project is meeting its requirements and enables the development team to trace defects and issues back to their source.
- Change Management: Requirements can change during the project’s development due to various factors such as evolving business needs, new regulations, or technology changes. Therefore, change management is critical to ensure that any modifications to the requirements are managed properly and communicated to all stakeholders.
In conclusion, getting the initial requirements right is crucial for the success of any data science project. Following the MIL-STD-498 standard and gathering the necessary information during the initiation of the project can help ensure that everyone is on the same page and that the project stays on track. A well-defined SRS can also serve as a reference throughout the project’s development, helping the team make informed decisions and avoid misunderstandings.