The HL7® FHIR® community has long relied on connectathons as crucibles of interoperability — intense, cooperative testing events where engineers huddle to make their systems talk to each other. But as FHIR adoption accelerates across healthcare, a question looms: how do we ensure every system can communicate with every other, not just in a two-day event, but in the messy, continuous reality of healthcare data exchange? This was the core theme of a recent WEDI webinar hosted by Drummond titled “Beyond Connectathons: How Scalable FHIR Testing Is Possible With FHIRplace.” The discussion brought together industry experts to explore how Drummond’s new FHIRplace initiative is pushing FHIR testing into a scalable, always-on era of interoperability.
Moderated by Timothy Bennett of Drummond Group, the panel featured Rob Brull (Infor), Mark Scrimshire (Onyx Health), and Michael Gould (ZeOmega) – each bringing a front-line perspective on FHIR implementation. Bennett provided insight from decades of interoperability testing experience, Brull spoke from the vantage point of an EHR platform provider, and Scrimshire and Gould represented the payer and population health solution angles, respectively. Their consensus? The industry’s current testing habits need a reboot to keep up with FHIR’s rapid rollout.
The Role and Reach of Connectathons in FHIR Testing
For years, HL7 FHIR connectathons have been the go-to venue to vet emerging FHIR implementations. These hackathon-style events gather developers from different organizations in one place to test their FHIR APIs against each other. Teams arrive with new FHIR servers or apps and spend a couple of days pairing up with other participants to exchange data and verify compatibility. Connectathons are invaluable for shaking out early bugs, clarifying ambiguous parts of the specifications, and forging relationships across organizations.
Yet, as the name implies, connectathons are short-lived. They typically focus on a narrow set of use-case scenarios or implementation guides agreed upon beforehand — for example, a connectathon might center on a Da Vinci Project use case, meaning many other FHIR interactions aren’t exercised. As Mark Scrimshire pointed out during the webinar, participants often stick to the “happy path” – the ideal interactions where everything goes right. Real-world interoperability is far messier: systems might send unexpected data, edge cases crop up, and not everyone follows the script. The result? Many integration issues only surface after a connectathon, during real deployments, simply because there was no time (or no partner available) to test every permutation at the event.
Organizations have tried to fill these gaps with pilot projects – limited-scope integrations with a partner to continue testing in a quasi-real environment. Rob Brull shared that his team frequently arranges one-off pilots after connectathons to iron out issues that never came up during the event. Pilots provide more realism than connectathons, but they are also slow and resource-intensive; you can only run so many before deadlines hit. This piecemeal approach doesn’t scale when dozens of connections and use cases need validation across an ever-expanding network of FHIR endpoints.
Why FHIR Testing Must Level Up
Today’s interoperability landscape is dramatically more complex than a few years ago. Thanks to initiatives like the CMS Interoperability and Patient Access rules, health plans must expose patient data via standardized FHIR APIs, and the ONC Health IT Certification Program now requires certified EHRs to include FHIR-based APIs. In practice, this means dozens of payer systems and EHR products are rolling out FHIR capabilities simultaneously. They all need to work together, but who is testing that at scale?
The webinar panelists underscored this pinch point. Michael Gould noted that the number of FHIR endpoints has exploded, making it impractical to vet each new interface through ad-hoc meetings or sequential pilots. Timothy Bennett suggested that the industry must move from artisanal, one-at-a-time testing to something more akin to an assembly line or continuous integration for FHIR. In other words, FHIR testing has to become scalable, automated, and collaborative by default. This is exactly the gap Drummond aims to fill with FHIRplace.
Introducing FHIRplace: Scaling Up with Full-Matrix Testing
FHIRplace is Drummond’s answer to the scalability problem—an automated interoperability testing platform that goes far beyond the confines of a one-off event. Bennett introduced FHIRplace as a way to bring “full-matrix” testing to FHIR. Full-matrix means that if ten systems participate, each one is systematically tested with all the others in all relevant scenarios, rather than just a few random pairings. It’s like turning a connectathon into an always-on, all-to-all testing environment.
How does it work? FHIRplace automates the heavy lifting of interoperability testing. It generates test cases, executes them across multiple systems, and analyzes the results. It provides a neutral sandbox where FHIR servers and client applications can connect and verify they truly speak the same language. Essentially, it’s a continuous virtual connectathon that can run anytime (not just during a conference) and cover far more ground. One panelist likened it to a stress test for FHIR interfaces. You subject your implementation to a battery of scenarios, including edge cases and error conditions, so that you uncover weaknesses in the lab rather than in production. If a connectathon is like a dress rehearsal, FHIRplace is a full-season practice schedule with the entire cast, ensuring everyone knows their lines before opening night.
Bennett highlighted that this full-matrix approach isn’t without precedent. Drummond has been orchestrating interoperability test events in other industries for over 25 years, even before FHIR existed. Those experiences—over 40 multi-vendor test events for past data standards—informed the design of FHIRplace, tailoring it to the intricacies of healthcare APIs. The platform can incorporate specific Implementation Guides and use cases, ranging from basic patient demographics to complex value-based care workflows, and verify each participant against the same rigorous criteria. All results are logged and shared so organizations can pinpoint where a breakdown occurred, whether it’s a missing field, a misunderstanding of the spec, or a performance bottleneck, and fix it before real patients or partners are impacted. The shared testing baseline also means everyone is held to the same standard, which promotes consistency and trust. No more “it worked in my test but not in yours” surprises.
In summary, the panel highlighted several ways a full-matrix platform like FHIRplace is transforming FHIR testing:
- Wider coverage of scenarios: Including edge cases and negative tests that rarely get exercised in connectathons, ensuring robust handling of real-world data quirks.
- Concurrent, collaborative testing: Multiple organizations can test together simultaneously, saving time compared to serial one-on-one pilot projects and fostering a spirit of transparency.
- Automated execution and analysis: Test suites run with minimal human effort, and results are analyzed centrally – reducing manual labor and human error in assessing interoperability.
- Consistent standards compliance: All participants are validated against the same playbook of FHIR profiles and workflows (e.g. checking conformance to a given implementation guide), which aligns interpretations of the spec and flags deviations early.
- Continuous availability: Instead of waiting months for the next connectathon, organizations can use FHIRplace on-demand or in regular intervals, integrating testing into their development cycle and ensuring readiness whenever a new partner or requirement comes along.
Achieving Interoperability Readiness for Go-Live
A powerful theme in the discussion was interoperability readiness – the confidence that when you flip the switch in production, your FHIR connections will just work. The panelists shared a vision where testing becomes a proactive exercise, not a reactive scramble. Mark Scrimshire emphasized testing not just the “sunny day” scenarios but also the rainy days and thunderstorms. In practice, that means deliberately simulating unexpected inputs and error conditions. With a platform like FHIRplace, this kind of exhaustive shake-down can happen early in the development lifecycle. It’s akin to using a flight simulator to train pilots: you can crash as many times as it takes in the simulator so that you never crash in real life. By running through countless what-if situations virtually, healthcare organizations can deploy new FHIR APIs or integrations with far greater peace of mind.
Rob Brull also noted that such rigorous pre-production testing has a side benefit: it builds trust among partners. When an EHR vendor, a payer, and a digital health app have all tested against each other (or against the same battery of tests) in FHIRplace, they gain mutual confidence. Each party knows the others have seen and passed the same interoperability gauntlet. That trust can accelerate onboarding new connections – fewer lengthy “let’s double-check X” meetings and more plug-and-play integration. This kind of collaboration is especially crucial as the industry inches toward true plug-and-play interoperability, where any certified system might connect to any other with minimal custom work.
A New Chapter in FHIR Testing
The insights from “Beyond Connectathons” make one thing clear: the future of FHIR interoperability testing will look very different from the past. Connectathons and bespoke pilots have served us well, but they alone cannot keep pace with the demand for seamless data exchange. Scalable, automated platforms like Drummond’s FHIRplace are poised to become essential tools for any organization serious about FHIR implementation quality. All of the panelists agreed that achieving FHIR’s promise of interoperability at scale requires moving beyond the heroic efforts of individual teams at quarterly events, toward a more continuous, community-driven approach.
In an era of expanding regulatory requirements and ever-more interconnected health systems, this evolution in testing is timely. It’s no longer just about can your system exchange data in theory; it’s about proving it can do so reliably with many partners in practice. By investing in full-matrix testing and collaborative platforms, healthcare IT teams can catch issues early, reduce integration costs, and deliver on the interoperability promises made to patients and stakeholders.
In short, it’s time to go beyond connectathons in more than just name. FHIRplace is reimagining what’s possible for scalable FHIR testing, turning what used to be occasional sprints into a sustained, high-quality marathon of interoperability. And as this webinar’s discussion illustrated, that shift isn’t just visionary – it’s entirely within reach, and it’s happening now.