Risk & Reward: Taking The Guesswork Out Of Compliance & Safety For Medical Devices

The world-famous and often quoted maxim: ‘the one constant is change’ certainly holds true for the medical devices sector. That doesn’t stop at the fast pace of innovation, but starts at the minefield of compliance and regulation that is rightly imposed on the sector that is expected to recover and reach $603.5 billion in 2023.

Needless to say, safeguards must be built into life-critical treatments, products and interventions, where a design flaw or unsufficient attention to potential harms can spell the difference between safe or potentially lethal medical devices.

[Read more…] about Risk & Reward: Taking The Guesswork Out Of Compliance & Safety For Medical Devices

Helping Medical Device Businesses To Balance Fast Development & Meet  Regulatory Frameworks

When does product development become a matter of life and death?

There are high stakes riding on the development of medical devices and manufacturers. Everyone from startups to global players often only get a limited opportunity to jump from prototype to a wider market acceptance. The price of failure can be punishing!

The development cycle of medical devices is very short – a third of the devices developed and sold are younger than three years.

[Read more…] about Helping Medical Device Businesses To Balance Fast Development & Meet Regulatory Frameworks

Agile Medical Device Development: Do or Don’t?
Content aspects

In this article we highlight two important differences between traditional organizations and agile organizations. These two content-oriented aspects demonstrate that working agile requires a different view and therefore a different approach to the same problem. These aspects are:

• “Continuish integration”
• Solution Intent

This article is the third of three articles explaining the differences between a traditional way of product development and a more agile way of product development. These articles are based on a survey conducted on agile product development with medical device companies. The previous articles on organizational aspects and process aspects can be found by following the links.

---

1.   “Continuish integration”

When developing medical devices the typical approach is based on the FDA guidance with regards to Design Controls, mostly executed as a stage gate process.

Following the stage gates, first ALL User Needs are identified, reviewed and approved, then ALL Design Inputs are identified, reviewed and approved, etc. As a result, the entire scope of the product is moved from stage gate to stage gate. Going back to a previous gate is considered to be a change and a formal change process needs to be followed. A change control board will either approve or reject the change. As a consequence, moving this large batch from gate to gate is a slow process and can take months and sometimes even years before any result is visible.

In an Agile way of working the aim is to work in small batches that can be delivered in months, weeks or even days, called Iterations or Sprints. Each iteration adds an increment of the scope to the product.

At the end of each iteration there is an integration point, where – at least in software development – all code from the different teams is integrated. This integration point is the forcing function that verifies changes and validates assumptions across the entire system. Development teams invest in automation and infrastructure that builds, integrates, and tests every developer change and provides nearly immediate feedback on errors. This is called continuous integration.

A medical device however does not only consist of software. Typically 3 elements play an important role:

1. Hardware
2. Software
3. Compliance (evidence to demonstrate regulatory compliance)

In Agile, the goal is to develop these aspects of a medical device in parallel, so at each integration point, hardware, software and compliance are in sync.

Systems, such as medical devices, that consist of hard- and software are far more difficult to continuously integrate because:

• Long lead-time hardware items may not be available
• Integration spans organizational boundaries
• Automation is rarely end-to-end
• The laws of physics dictate certain limits

Therefore “continuish integration” is suggested. The goal is frequent partial integration with at least one full solution integration each program increment (every quarter).

When developing medical devices, verification and validation (V&V) practices are an integral part of the quality and compliance process. V&V commonly occurs in a testing phase once the solution is completely developed. Instead of this work being performed separately following development, the Agile best practice is to integrate this work into the effort required to bring each Feature of the product to completion.

 

---

2.   Solution Intent

Once people hear about the Agile value “working software over comprehensive documentation” for the first time, they seem to always remember that Agile is equal to not working with requirements. That’s a misunderstanding since it’s important to store, manage, and communicate the knowledge of current and intended behavior of the medical device. This is called Solution Intent.  In Agile product development the Solution Intent includes both fixed and variable specifications and designs; reference to applicable standards, system models, and functional and nonfunctional tests; and traceability. A different approach from traditional product development where specifications need to be fixed up front.

Solution intent stores, manages, and communicates ‘what is being built’ and ‘how it will be built.’ It serves many purposes:

• Provides a single source of truth regarding the intended and actual behavior of the solution
• Records and communicates requirements, design, and system architecture decisions
• Facilitates further exploration and analysis activities
• Aligns the Customer, Agile Teams, and Suppliers to a common mission and purpose
• Supports Compliance and contractual obligations

Fixed intent represents the ‘knowns.’ They may be nonnegotiable, or they may have emerged during the course of development.

Variable intent represents the elements that allow the exploration of the economic trade-offs of requirements and design alternatives that could meet the need. Once established, these new insights will eventually become fixed requirements. Moving from variable to fixed requires gaining knowledge to make decisions.

An example of an excellent solution to store, manage and communicate the Solution Intent of a medical device is Cognition’s Guided Compliance solution called Cockpit.

An excellent framework to gain knowledge and make design decisions is the Rapid Learning Cycles framework. It implements the same iterative approach as Agile does. However where Agile focuses on delivering working product increments, Rapid Learning Cycles focuses on delivering knowledge. Closing Knowledge Gaps in short iterations – for example by doing a set of experiments – enables to make well substantiated Key Decisions with regard to alternative solutions.

Requirements and designs used for verification and validation are also captured in the Solution Intent repository.

• Verification determines that the current system increment was built according to the specifications captured in the backlog, and in alignment with the Solution Intent (i.e., we built the solution right).
• Validation determines if the increment’s backlog items meet the system’s fitness for use or Intended Use. (i.e., we built the right solution).

Traceability within the Solution Intent ensures the artifacts produced each increment (software, hardware components, etc.) address regulatory and compliance specifications, providing end-to-end traceability (evidence) that V&V requirements have been met.

Conclusion

Making the shift to “continuish integration” and implementing a Solution Intent repository is not a trivial effort. Two strategies help mitigate barriers to change.

First, many of the checks can be automated. Incorporating compliance tests into the automated build process ensures that the build does not pass unless the feature passes its compliance requirements. (And yes, even hardware-based systems can incorporate automation in the testing process.) This reduces manual verification and frees quality and compliance personnel to focus on higher-level activities.

Second, shifting to continuous V&V can be incrementally tested and refined in pilot projects—before the approach is implemented across all agency programs.

Agile Medical Device Development: Do or Don’t?
Process aspects

In this article we highlight two important differences between traditional organizations and agile organizations. These two process-oriented aspects demonstrate that working agile requires a different view and therefore a different approach to the same problem. These aspects are:

• Flow
• Cadence and Synchronization

This article is the second of three articles explaining the differences between a traditional way of product development and a more agile way of product development. These articles are based on a survey conducted on agile product development with medical device companies. The previous article on organizational aspect can be found here.

---

1.   Flow or slow

A couple of weeks ago I was on my way from the airport to a conference venue, where I was up to present at 9:30 hrs. In the taxi I asked the driver how long it would take to get to the venue. He answered that it is very busy on the road with several traffic jams, so his estimate was that it would take about 50 minutes to get there.
The only thing the taxi driver mentioned was the traffic jams (delays). He did not mention what the top speed of the car is or what the maximum speed on the highway is. He did not mention that he is an amateur race driver with good driving skills. He didn’t mention the road conditions – wet or dry tarmac – Why did he not mention any of this?
The taxi driver knew from experience that none of these aspects would make a substantial difference to the overall time to get to the venue. He knew that the traffic jams – delays – are the main influence on the time to reach the destination.

What is lead time?

Simply said: Lead Time = Waiting Time + Processing Time.

In the example of the taxi driver the lead time is the actual time from getting into the taxi at the airport to getting out of the taxi at the venue. The processing time is the time the taxi is actually driving, and the waiting time is the time the taxi is standing still as a result of traffic jams.

Value Stream mapping

Let’s take a deeper look at applying lead time to business processes. As an example, you see below the process and steps needed to go from a request to implement a feature in an application to the actual deployment of it.

The above example clearly shows Waiting Time and Processing Time. The total processing time is (1 + 1 + 1 + 4 + 1 + 3) 11 hours. The total waiting time is (1 + 2 + 2 + 1 + 1) 7 weeks (280 hours). The total lead time is 280 + 11 = 291 hours and only 11 hours are needed to do the job. This results in a time efficiency of less than 4%.

The above exercise is called “Value Stream Mapping”. It is a systematic way to visualize all the steps in a process to deliver value. And as shown in the above example, most of the time in your business processes is waiting time.

How to improve flow?

Looking at the above example, what would you do to improve the process? You have 2 options to choose from: improve processing time or reduce waiting time. Let’s examine both.

Improve processing time

Let’s pick the activity with the longest processing time: “Code & test”, which takes 3 hours. Assume we realize a massive improvement and reduce it with 1 hour (33%). As a result, the lead time is reduced from 291 hours to 290.

Reduce waiting time

Let’s pick the longest waiting time, for example “E-mail Tech Lead” (and wait for an architect to become available), which takes 2 weeks (80 hours). Again, assume that we realize a massive improvement and reduce it with 27 hours (33%). As a result, the lead time is reduced from 291 to 264, a much bigger improvement than achieved by focusing on improving the processing time.

Accuracy & completeness

There is one more aspect that you may want to take into account and that is Accuracy & Completeness of the work done during processing time. If the work is not accurate and/or incomplete, it prevents the next step in the process to start, causing both rework (extra processing time) and as a result additional waiting time.

Conclusion

In traditional organizations you typically see that the focus to improve the overall process is on reducing the processing time. The most commonly used improvement is to add more resources to the task – see previous article about hand-over time.

In agile organizations we believe that people are already doing the best they can, but their performance is limited by the system itself – the waiting times in business processes.
If you want to improve your business processes, focus on reducing waiting time instead of processing time. You will improve flow so much more.

Improving flow brings another advantage to the table. It enables earlier feedback and as a result provides the opportunity to learn and make adjustments. In agile, learning is an integral part of the process. In contradiction with a traditional process where a lessons learned is done as part of decommissioning the project – to produce a report that seldomly gets read, let alone apply the recommendations –, agile has implemented learning at each iteration, enabling the continuous improvement of the teams and the solution they build.

---

2.   Cadence, synchronization and cross-domain planning

Every development of a new or improved medical device starts with a plan. As part of the FDA’s guidance (FDA 21 CFR 820.30 (b) and ISO 13485) a design and development plan is required.
What we see happening in many organizations is that the project manager creates the plan at the start of the project. The plan is worked out in a great level of detail, so the plan is regarded thorough and well thought-out.

The plan

I don’t know about your plans, but mine seldom go the way I was expecting them to go. Along the way, all kinds of things pop-up which I didn’t foresee at the start of the project. And this isn’t strange because at the beginning of a project you have limited knowledge and experience. You may have a few hypotheses and assumptions, with no evidence or assurance that these are achievable. In terms of deadline, number of resources required, total cost, etc. anything you provide is an estimation, however management will often interpret these as certainties and commitments, causing a rigid control of the plan and eliminating any flexibility.

An administrative nightmare and low commitment

The other thing we see is that the project manager creates and manages the plan. He will ask input from all stakeholders and may even involve them in the initial plan. In addition, he works out the plan in a lot of detail, creating the impression that everything is well thought-out and carved in stone.

There are a couple of problems with this approach. As the project manager creates and manages the plan, it will be HIS plan. The teams in the project will feel this as something that has been forced on them. It isn’t their plan and as a result they will feel less committed to it.
In addition, because the plan is so detailed, it becomes an administrative nightmare to keep it up to date with reality. I have seen cases where project managers have stopped updating the plan, due to the number and frequency of changes. It’s hard to stay on track when the track is gone.

Planning is a regular effort – not one-time only

In a Lean-Agile world a different approach is taken towards planning. First, Lean-Agile takes changes to the plan as a blessing instead of a curse. Change is based on new insights and learnings and therefore an opportunity to steer into the right direction and make the outcome better. In order to achieve this, Lean-Agile sees planning as a regular activity that is executed in a certain cadence – for example every 10 – 12 weeks (quarterly). This enables a review of the previous phase and learn from it. In addition, it enables to adjust because of new developments and/or priorities.

Planning is a team effort – not one-person only

In Lean-Agile the plan is not owned by the project manager; it is made and owned by the teams. In a cross-domain, big room planning meeting, all team members (as much as possible) and stakeholders are present to make a plan for the next 10 – 12 weeks. The teams base their plans on their estimation of the size of the work and the capacity they have available to do the work (velocity). In addition, they will identify cross-team dependencies and resolve those through face-to-face contact between the teams involved. As a result, they synchronize their team-plans to ensure the overall plan is achievable. Finally, they will identify risks and mitigations to minimize impact on the plan.

When planning becomes a team effort, the teams own the plan and are much more committed to deliver according to their plan. In practice we see the teams overachieving on their plans.

Conclusion

Cadence creates predictability and provides a rhythm for the organization. Synchronization causes multiple perspectives to be understood, resolved, and integrated at the same time. Applying cadence and synchronization, coupled with periodic cross-domain planning, provides the tools needed to operate effectively in the presence of uncertainty, inherent in medical device development.

Agile Medical Device Development: Do or Don’t?
Organizational aspects

In this article we are highlighting two important differences between traditional organizations and agile organizations. These two aspects demonstrate that agile requires a different view and therefore a different approach to the same problem. These aspects are:

1. Resource allocation
2. Organizational structure

This article is the first of three articles explaining the differences between a traditional way of product development and a more agile way of product development. These articles are based on a survey we did on agile product development with medical device companies.

---

1. Resource Allocation

Imagine you are the project manager on a large project and need an additional resource to pick up extra work. You probably must go through some kind of approval process, because someone – probably a departmental manager – must make a person available and that is difficult because everybody is busy on other projects.

FTE versus Individual

What we see happening a lot in organizations is that there is no dedicated person available for the request at hand. As a result, an FTE (Full Time Equivalent) is made available, basically several part time people: John is available on Mondays, Lucy on Tuesdays and Thursdays. Tony can handle the Wednesday and Anna will cover the Fridays.

In fact, what happens is that you as a project manager receive an arm from one person, a couple of legs from another, the body from someone else and the head from a fourth person.

Basically, you end up with an additional problem in stead of a solution. These people must work together, must communicate and hand over tasks. You were probably asking for an individual and not an FTE. In addition, for knowledge workers – and much of our work is knowledge related – it is extremely hard to switch between multiple project contexts. It creates waste.

The waste of project switching

Gerald Weinberg introduced a rule of thumb to calculate the waste caused by project switching: With every additional project, 20% waste is added – view graph below.

Often management thinks the resource request is adequately covered and they calculate from a productivity point of view as if there is a dedicated FTE allocated. But in fact, they have created more problems than they can imagine. Not only the project that the resource is newly allocated to is negatively impacted from a productivity point of view. In addition, the other projects that these 4 resources are working on simultaneously are also negatively impacted! The root cause is not having a dedicated individual assigned to the project/team.

The result is that the overall productivity will decrease instead of increase.

Dedicated resources and teams

Agile suggests a completely different approach and the above issue with fragmented resources is one of the reasons. Dedicated teams and dedicated resources in these teams are a pre-condition to aim for high velocity and high quality output of teams.

Teams can meet their responsibilities only through constant communication and collaboration, and through fast, effective, and empowered decision-making. Relationships within the team are therefore based on trust, facilitated by a common mission, iteration goals, and team objectives. This requires teams to stay together for a longer period with dedicated team members.

Prioritization

In an organization where resources are spread across multiple projects, the need to make hard decisions about priorities is less present. The symptom usually heard is that projects take forever to finish. The root cause is that people and teams are not dedicated working on a single objective.

When teams and resources are dedicated, management needs to step up and make clear decisions about priorities. More importantly, they must make decisions on what work not to do or do later, when resources become available.

---

2. Organizational structure

Traditionally, organizations are functionally structured. In such an organization, software developers work with software developers, testers work with testers, systems engineers and architects work with each other and operations work with themselves. This works well for people management and it’s easier to control and optimize individual silos.

However, the value delivered to a customer doesn’t flow quickly, as it must go through all of the silos. Value delivery is inhibited by hand-offs and delays between the different silos. Political boundaries and misalignment in targets and goals across the silos prevent collaboration. In addition silos may encourage geographic distribution of functions which also makes communication across silos more difficult especially when different time zones are involved. The daily involvement of managers and project managers is necessary to move the work across the silos. As a result, progress is slow.

The Agile organization – Team of teams

Instead the Scaled Agile Framework (SAFe^®) suggests setting up a cross-functional organization that optimizes the full value stream facilitating the flow of value.

To enable this cross-functional organization to deliver value, it has all the resources – everyone and everything – to define, deliver and operate solutions. In addition, it is self-organizing and self-managing and doesn’t require the daily task and project management no longer. As a result, value flows more quickly, with a minimum of overhead. SAFe calls this an Agile Release Train.

Agile Teams

The Agile organization is built on Agile teams, which are also cross-functional and consist of 5 to 11 people, who define, build, test, deploy and deliver an increment of the solution in a short iteration (every 2 to 4 weeks).

Agile Teams do not operate independently. They plan together, integrate, demo, deploy and release together, and learn together. All teams operate within a common framework that governs and guides them as one organization – The Agile Release Train.

The above findings are based on a recent survey among medical device development professionals, where we tried to find an answer to the question “Agile Medical Device Development: Do or Don’t”. In a follow-up of 3 articles we provide 6 aspects of working agile that differ significantly from working project-based. This article is the first one, focusing on the organizational differences compared to traditional organizations. Watch our web site for the next 2 articles.

Agile Medical Device Development: Do or Don’t?

Today, software development is predominantly applying Agile practices. Scaling agile from a couple of teams to hundreds or even thousands of people, developing software in different locations world-wide is today’s challenge.

Medical devices however do not only consist of software. Hardware is also an important element to develop and so is compliancy to regulatory standards. Compliancy in the form of a Design History File (DHF) and a Quality Management System (QMS) are important deliverables to demonstrate compliancy and “must-haves” in releasing the device to the market. The question arises whether Agile practices can also improve the development of hardware and compliancy dossiers.

In a recent survey among medical device development professionals, we tried to find an answer to the question “Agile Medical Device Development: Do or Don’t”.

The following questions were asked in the survey:

1. Are you familiar with Agile?
2. Are you familiar with the Scaled Agile Framework (SAFe ®)?
3. Agile development for medical devices can be successfully applied to?
4. What is the biggest challenge to adopt Agile in your organization?
5. Do you think Agile can help improve the adoption of engineering tools?

Are you familiar with Agile?

The survey shows that the majority of the medical device development professionals are familiar with Agile and apply it in their day to day activities. Many of the organizations are applying Agile only to software development. In total you can conclude that the vast majority – 82% – of medical device professionals apply Agile methods & practices.

 

 

 

 

---

Agile at team level – Scrum

Among the medical device professionals, Scrum is predominantly used as an Agile method (78%). Scrum is an agile method to implement agile at a team level. As a result, we can conclude that agile is mostly used at team level and has not been scaled to implement cross-domain alignment and collaboration.

 

 

 

---

Are you familiar with the Scaled Agile Framework (SAFe ®)?

The Scaled Agile Framework (SAFe®) is one of the most adopted frameworks to scale Agile development to a large number of teams and ensure a cross-team cadence and synchronization.

No Agile at Scale

Looking at the usage of one of the most popular agile scaling frameworks, SAFe®, none of the survey participants has implemented this framework. Moreover, 75% of the attendees has never heard about SAFe and only 25% has heard about. The conclusion is that medical device development is not yet applying scaled agile methods and practices to align a large number of teams during the development process.

 

---

Agile development for medical devices can be successfully applied to …?

The previous questions were predominantly focused on identifying to what extent Agile has entered the development processes for medical devices.
The next questions are much more focused on identifying what the respondents think is possible in the future with regards to adopting and implementing Agile methods and practices in their organization.

Agile all the way

It is therefore interesting to see that 64% of the respondents think that Agile can be successfully implemented for all aspects of medical device development, including software, hardware and compliancy (Design History File and Quality Management System).

Currently, most of the respondents still use waterfall-oriented methods to develop the compliancy deliverables. One of the respondents said, that this results in a bottleneck to adopt Agile across all aspects of medical device development. If you are delivering the product (e.g. compliancy deliverables) in a waterfall-oriented method, it is hard to profit from the benefits of Agile development. You must take the whole value stream into account.

---

What is the biggest challenge to adopt Agile in your organization?

To the question what the biggest challenge is to adopt Agile in their organizations, more than half of the respondents view management support and endorsement of Agile as the biggest hurdle.

Removing the silos and creating cross-functional teams is the second largest challenge within medical device development. This one is related to the first challenge, because if management doesn’t endorse Agile, they are not open to change the organization from a control-oriented structure to a value-driven structure.

Interesting to see is that the respondents don’t see combining software and hardware development as a challenge at all. This seems to contradict with the second biggest challenge of breaking the silos and building cross-functional teams. However, in the discussion with the respondents it appeared that integrating other disciplines in the agile process, such as compliancy (DHF & QMS), is much more challenging.

---

Do you think Agile can help improve the adoption of engineering tools?

In discussions with the respondents it became clear the adoption of Product Line Engineering tools in organizations has many challenges. But they think that adopting a common process, such as Agile, could improve the adoption of tools to support and drive the product development process.

Engineers love their tools and don’t like to switch to something else, unless it makes their job a lot easier. The issue with this is that a lot of engineers are sticking do their own tools not realizing that this creates lots of delays and rework during the total development cycle of medical devices.
In addition, it makes the creation of a consistent Design History File a challenging job with a lot of converting and reformatting of information, delaying the time-to-market of the medical device.

 

---

What is required to successfully apply Agile to medical device development?

To successfully transform from a traditional project-oriented organization to an agile development organization requires some fundamental changes which truly take another approach to the same problem. These changes may feel like the world upside down comparing to what is used to be done in a traditional project organization. These mindset changes can be divided into three groups: organization, process and content.

1. Organization
   • Resource allocation
   • Organizational structure
2. Process
   • Flow of value
   • Cross-team collaboration
3. Content
   • Continu-ish integration
   • Solution intent

Each mindset change is explained in 3 follow-up articles, which will appear in this blog area, so stay tuned.