Performance measures · Geospatial modeling · Evaluation · Systems integration · Information security
EMS program evaluation and data: evidence a funder will accept
Most rural emergency medical services (EMS) programs that work cannot prove it. The service knows the program is helping, and the funder asks for outcomes. But nobody defined the denominator, the comparison or the measure before launch, so the data to answer the question was never collected. The Paramedic Foundation (TPF) works the other way around. It builds the measures first, runs the analysis to a standard that holds up, and leaves a system your own people can run.
What TPF does with data
Performance measure development. TPF helps decide what a system should measure and what it should stop measuring. Response-time percentiles answer a narrow question and have crowded out most others. The useful measures are usually about clinical outcome, the right destination, a workforce that lasts, and cost avoided.
Coverage and deployment modeling. Map-based analysis of where units are, where calls come from, and what a change to either would do. This analysis sits behind a standards-of-cover document, a station move or a regional merger. It also answers whether a service can keep a second unit going.
Patient care record and data systems. Work with the National EMS Information System (NEMSIS) and electronic patient care records, and checks on data quality. It includes the slow work of finding out why a field is empty in 40% of records before drawing a conclusion from it.
Dashboards and data governance. TPF builds reports that a board, a council or a state office will actually read. It also sets the rules on who owns the data, who may see it and how long it is kept.
Program evaluation. Logic models, outcome tracking, and models of cost avoided and return on investment. Where the question is more than a numbers question, TPF uses mixed methods. That includes qualitative and community-engaged research, designed and led by people who have run such studies. Each design states a primary outcome, a defined denominator and an analysis plan set in advance. Each is reported against the guidance that fits the claim.
Applied health services research. Peer-reviewed work on rural EMS access, community paramedicine outcomes and health economics. TPF does this work with academic partners and publishes it rather than filing it away.
Systems integration. Health care data standards such as Fast Healthcare Interoperability Resources (FHIR) and Health Level Seven (HL7), and hospital admission, discharge and transfer feeds. Links to computer-aided dispatch and geographic information systems, and dashboard platforms. This is what makes an analysis repeatable, instead of a one-off extract someone pulled by hand.
EMS program evaluation: the measurement comes first
Order matters more than any single method. A program two years old cannot go back and collect what it never captured. The most common job TPF is asked to do is repair. A typical case is a community paramedicine program with real results but no evidence a payer or funder will accept, because the evaluation plan was an afterthought.
Getting the order right is not expensive. Before launch, the program decides what outcome it is trying to change and what it will compare against. It sets the denominator, the fields it must capture to answer the question, and who is responsible for capturing them. That conversation takes a day. It can decide whether a program renews or ends when the grant does.
It also keeps the claims honest. TPF separates figures a program reports about itself from evidence at the level of an evaluation, and says which it is citing. A service reporting its own avoided transports is useful. It is still different from an independent evaluation. A report that blurs the two will not survive a reviewer who notices.
How TPF protects project data
Agencies now often ask how data is handled before they ask about methods. A consultant who cannot answer precisely does not get the project. TPF operates a Microsoft 365 environment governed by Microsoft Entra ID access controls. It maintains organizational alignment with ISO/IEC 27001:2022 through an annual self-audit framework. Under Clause 5.3, a designated Security and Privacy Officer is responsible for data governance, incident response and regulatory compliance monitoring across projects.
Project data is sorted into three classes: Public, Internal Use Only and Confidential. Only named, authorized people on each project can see it, and it moves only through approved secure channels. Some projects involve Criminal Justice Information Services data, or participants in the European Economic Area and the United Kingdom. For those, TPF keeps extra sets of contract clauses under its related framework governance.
When a project involves protected health information, the duties come from the contract and are handled as a matter of readiness. The agreement decides what applies, and TPF scopes the data handling to it before the work begins.
Who does this work
The specialists TPF calls on for data and evaluation
EMS data and analytics, program evaluation and applied research, and systems integration are three of the fields where TPF keeps a standing group of specialists. Projects in this area of work are staffed from the fields below and scoped to the work rather than to a fixed roster.
The fields include EMS performance measures, map-based analysis, coverage modeling and standards of cover. They include health services research, continuous quality improvement, health economics and rural health research. Others are program evaluation design and reporting, and qualitative, mixed-methods and community-engaged research. The list also covers health care data standards and systems integration, dashboards and data governance, and information security and access control.
Named profiles, credentials and current roles for all of the specialists TPF calls on are on the team page. TPF names the people proposed for a project in the proposal for that project, where the names can be matched to the scope.
Published work and public tools
Selected peer-reviewed publications by TPF principals and subject-matter experts are listed on the publications page. Free financial and staffing models from TPF are on the financial modeling page. They include the EMS staffing and schedule calculator and rural cost and revenue models. The reference library and the community paramedicine historical archive hold the source material behind much of this work.
About data and evaluation engagements
Common questions
Why does an evaluation framework need to be built before a program launches?
The measures decide what data the program collects. A program that has run for two years cannot go back and collect what it never captured. The most common failure TPF is asked to repair is a program with real results and no evidence a funder will accept. Nobody defined the denominator, the comparison or the outcome before the program started.
What information security standard does TPF work to?
TPF operates a Microsoft 365 environment governed by Microsoft Entra ID access controls. It maintains organizational alignment with ISO/IEC 27001:2022 through an annual self-audit framework. Project data is sorted into three classes: Public, Internal Use Only and Confidential. Only named, authorized people on each project can see it. Extra sets of contract clauses cover projects that involve Criminal Justice Information Services data or participants in the European Economic Area and the United Kingdom.
Does TPF do peer-reviewed research or only consulting analysis?
Both, and TPF keeps them apart. Consulting deliverables state the design, the measures and the limits openly so the analysis can be checked. They are written and cited in American Psychological Association style, 7th edition (APA 7). APA 7 governs the writing, not the method. Reporting follows the guidance that fits the claim, such as CHEERS 2022 for an economic evaluation or the STRESS guidelines for a simulation study. TPF principals and subject-matter experts also publish in peer-reviewed journals. TPF does not present a consulting report as a study, or a study as a consulting recommendation.
Talk to TPF about evaluating your EMS program
If your program has not launched, the measures can still be designed properly. If it has, TPF can show what the existing data will and will not support before anyone promises a funder something it cannot deliver.