> For the complete documentation index, see [llms.txt](https://docs.data.ca.gov/odi-data-services-dif/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.data.ca.gov/odi-data-services-dif/modern-data-stack-accelerator.md). # Modern Data Stack Accelerator {% hint style="success" icon="calendar-circle-plus" %} [Book an office hour appointment](https://scheduler.zoom.us/d/2gelv5jh/office-hours-with-odi-s-data-services-team) to discuss if the Modern Data Stack Accelerator or other service offerings are right for you. {% endhint %} ## Overview The Modern Data Stack Accelerator (MDSA) is a practical approach to help departments rapidly adopt modern cloud-based data tools while working on a data challenge they want to solve.

Move from siloed data to integrated data in order to build trust in your organization's data

Your department may struggle to combine data across systems and/or continue to rely on painful and manual processes to do so. This manual effort costs time and diverts staff from higher value work. Or your department may be struggling with how to best acquire and use modern data tools. The Modern Data Stack Accelerator will help empower your team to use modern data tools effectively. Our goal is to demystify what building a modern data stack means, including: 1. architecting and procuring data stack components, 2. creating a culture of data operations across a team, 3. developing repeatable, automated, and observable processes, and 4. staffing to maintain a modern data stack. This service relies on advances in cloud data warehouses (e.g. Snowflake, Google BigQuery, Microsoft Azure, Redshift, etc) and complementary data services that make it easier, faster, and cheaper to provide data that is highly available, robust, and agile. This accelerator **starts with a real business problem** where supporting data has traditionally been difficult to combine and/or clean up. Using modern data tools in our environment, combined with training and consultation, your team will learn how to transform and automate this process. The goal of this service is to build your team’s capacity to use the modern data stack and be ready to move from demonstration to production. **This is a new service and our team is looking for partners willing to experiment and learn with us.** {% hint style="info" icon="book-open-cover" %} Read about recent Modern Data Stack Accelerator case studies published on [ODI’s website](https://innovation.ca.gov/our-work/projects/#page-1), including projects focused on: * [A data-driven approach to detecting cash benefit theft](https://innovation.ca.gov/our-work/projects/data-driven-approach-detecting-cash-benefit-theft/) * [Unlocking state hiring insights](https://innovation.ca.gov/our-work/projects/unlocking-state-hiring-insights/) * [Enabling analysis of California’s hiring and recruitment data](https://hub.innovation.ca.gov/data/enabling-analysis-californias-hiring-recruitment-data/index.html) {% endhint %} ## How this service can help your department This service can help your department tackle difficulties with: * Combining multiple datasets efficiently and automatically for analysis and reporting * Automating manual data quality efforts for high visibility analyses or reporting * Moving, querying, and analyzing large datasets * Assessing or trying new data tools to see if they work for your before making an investment
Four panels with icons representing the preceding list of difficulties the Modern Data Stack Accelerator can help with. There is one panel for each difficulty on a dark blue background with the same text as in the above list items.
## Who is a good fit for this service This service requires participation from your data team, IT, and a program area. This is a holistic service that works across your department teams to empower modern data use. This service is a good fit for organizations that have a real data question to address and: 1. have been struggling with developing trustworthy sources of data and are ready to try something new, 2. are open to consciously experimenting and learning together with CalData staff, and 3. are interested in or have already started investing in cloud data warehouses (like Snowflake, Google BigQuery, Microsoft Azure or Redshift to name a few)
Three panels representing the 3 listed items above. From left to right, first with an icon of a person pushing a boulder up a bar chart, the second a beaker and test tube, and third a database with a cloud behind it.
This service would **NOT** be a good fit for: * **Programs looking to set up technology without their organization’s IT support.** The Office of Data and Innovation (ODI) is helping you to identify solutions that can be embraced by the organization. Our service is to accelerate learning and adoption, not to provide ongoing support. * **Organizations that want to outsource their data work.** Adopting the modern data stack is more than just tools. While you can use systems integrators and contractors, the focus of this service is to empower state staff with improved access to data and modern tools to work with that data. * **Organizations that don’t have authority to access or use the data in question.** If you depend on data from another organization, that’s okay. You just need to be certain that it can be shared as part of the project. See the [Interagency Data Exchange Agreement (IDEA) ](https://docs.data.ca.gov/interagency-data-exchange-idea-guidebook/)for more on data sharing agreements. {% hint style="success" icon="calendar-circle-plus" %} If you’re unsure about your fit for this accelerator, [book office hours](https://scheduler.zoom.us/d/2gelv5jh/office-hours-with-odi-s-data-services-team) to discuss. We’re happy to advise! {% endhint %} ## What this service includes
A process flow diagram ordered from left to right. Starting with discovery sprint, then onboarding and training, then development sprints, then project retro, and finally handoff. These are then described below in text.

The Modern Data Stack Accelerator follows a structured process that allows for iteration and learning.

The engagement (4-6 months) will follow the following structure: | | | | ---------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Discovery and Planning | Discovery may include interviews, data sample reviews, and workshops to refine project scope and analysis questions. This culminates in a signed project charter. | | Foundational Training | Staff will receive baseline training in core concepts around the modern data stack. These are designed to level set across the partner team. | | Development Sprints | A number of 2 week periods of development. The specific number of sprints will be determined during discovery and planning. | | Handoff | Based on a plan developed with the department partner, ODI works with you on a handoff to move from demonstration to production in your environment. | | Project Retro | A look back at the project to identify what worked and what needs improvement overall. | ## Expectations of department partners The Modern Data Stack Accelerator (MDSA) helps departments build the skills, processes, and technical foundations needed to solve data challenges using modern data practices. ODI provides coaching, training, and technical guidance, but departments are expected to own and sustain the solutions developed through the engagement. Departments participating in MDSA should be prepared to: * **Assemble a cross-functional team** that includes program, IT, and data leadership, along with staff responsible for data stewardship, analysis, engineering, and system administration. Individuals may fulfill multiple roles where appropriate. * **Designate a Project Champion** who can coordinate the project, maintain momentum, and serve as the primary liaison between the department and ODI. * **Ensure leadership participation** from program, IT, and data leaders to help guide the project, remove barriers, and participate in key meetings. * **Commit staff time** for training, project work, and regular engagement activities over the course of the 4–6 month accelerator. * **Provide staff with foundational technical skills** relevant to the project, including basic SQL proficiency for analysts and data practitioners; engineering staff may also need experience with Python and cloud-based tools depending on project scope. * **Provide access to relevant data** and ensure the department has the authority and ability to share that data with ODI staff. * **Partner closely with IT** to support data access, infrastructure needs, and long-term sustainability of project outcomes. * **Support documentation and knowledge sharing** so lessons learned and successful practices can benefit other departments and jurisdictions. {% hint style="success" icon="calendar-circle-plus" %} If you’re unsure if you have coverage of these roles or any clarifications, [book an office hour](https://scheduler.zoom.us/d/2gelv5jh/office-hours-with-odi-s-data-services-team) to discuss. We’re happy to advise and suggest alternatives that may work for all of us. {% endhint %} --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter: ``` GET https://docs.data.ca.gov/odi-data-services-dif/modern-data-stack-accelerator.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.