When we designed the Opentrons Protocol API, we first and most importantly intended it to match the layout of the robot, and how our users would build a mental model of the robot's actions. It worked well, and in the time since we introduced it, scientists the world over have used the protocol API and the OT-2 to get a lot done.
In that time since we introduced it, however, we've also come to learn more about the downsides of its structure. These downsides come in three main forms:
- Complexity and inconsistency
- Maintainability and reliability
- Suitability for all purposes
To fix these problems, and provide a more reliable, solid, and user-friendly experience for our users that allows you to do science instead of robot management, we've created a new API: Opentrons Protocol API Version 2. This new API version is available in Robot Software version 3.14.0 as an open beta. You can use it by setting the "Use Protocol API V2" feature flag in the robot pane:
You interact with Protocol APIv2 in a different way in your protocols; documentation is available on docs.opentrons.com, and a helpful guide to migrating protocols from APIv1 to APIv2 is available here.
Why a new API?
It's a reasonable question to ask: you're used to the Protocol API as currently implemented, and it works for you; sure, there's issues you have to work around occasionally, but on the whole it's a good tool. Why change?
Complexity and Inconsistency
At first glance, version 1 of the protocol API is dead simple. There's a set of objects that you import at the beginning of your protocol:
modules . You then use a couple specific functions to load labware, instruments, and modules. Finally, you can use the objects you've loaded to control the robot. Not bad!
This functionality, however, is not really consistent with how many other Python modules work, and it's not consistent internally. In Python, you
import Python modules. These aren't modules - they're special objects designed to look like modules, but hook internally into each other. Some of them are objects that represent physical realities, like the
robot . Others are collections of functionality to inform the robot of new hardware, like
instruments . The objects these return aren't from the modules
instruments because there are no python modules called
instruments (well, there's one called
instruments , but you don't get to see it without jumping through some serious hoops).
The complexity becomes apparent when you start interacting further with the API. For instance, consider
aspirate . This function takes only three parameters:
location , and
rate . But it also tries to guess which parameters you're using if you don't specify: it tries to handle
aspirate(50, plate['A1']) , and
aspirate() , and
aspirate(50) , and
aspirate(plate['A1']) . Other functions do the same thing. In many cases, the only way to see that you made a mistake in calling the function is that the run log looks odd after you simulate it - and for many protocols, those run logs can be things you really don't want to scroll all the way through.
There's not really one single thing to point to about Protocol API v1 that exemplifies these problems, but there are many small examples, all adding up to a strange and inconsistent API that our users must learn by rote - a serious disservice to the types of smart, highly educated people who use our robots.
Maintainability and Reliability
When we write software, sometimes we mess up. That's a universal truth of software development. And the way to mitigate those mistakes, to catch them quickly - before they get to users - and to fix them easily is to write software in a way that is maintainable. That means clear, easy to understand, and simple code. This aspect is especially important in an open-source project, like all of our shipping code here at Opentrons, because these attributes are also what makes a codebase accessible to open-source contributors. Unfortunately, this is not true of the Opentrons Protocol API v1.
Codepaths are strange and circular (for an example, try figuring out what happens in
Pipette.move_to ). Handling of motion and coordinate systems is extremely complex, and hard to debug (for an example, try getting the exact coordinates of the tip of the pipette). All of this adds up to difficulty tracking down even the simplest bugs and inconsistencies when the robot is executing a protocol - exactly the kind of bugs that most frustrate our users, who want to be doing science, not fighting with a robot that won't behave.
Suitability for All Purposes
Our Protocol API was first written as an API for protocols to use to control an OT-2 (or, really, an OT-1). All of its interaction surface is designed with this in mind. It is designed and optimized specifically for this use.
Unfortunately, this isn't the only current use of the protocol API - far from it. In addition to controlling a robot executing a protocol, it is also used by
- The server software that controls the robot when it's not executing a protocol
- The layers of software that simulate protocols before running them to check for problems
- Any other parts of our software stack that want to parse protocols, like the Protocol Library
For these uses, the protocol API is a very bad match. Using an API in a way for which it was not originally intended is often the cause of many subtle bugs, and so it is in our case. It makes software hard to maintain, since you're always thinking around the intended use case of the API.
In addition, Protocol APIv1 is written such that the protocol is in charge of execution. The protocol is executed like a script and controls the robot directly. This requires all sorts of odd workarounds when a separate process (like the server process that controls the robot) is in fact executing the protocol in turn; it leads to race conditions that plague the reliability of things like
resume , and it leads to pollutions of state like issues surrounding the robot incorrectly thinking a tip is attached to the pipette when a protocol starts.
How does APIv2 fix these problems?
Unlike V1, Protocol API V2 makes it more obvious that the robot server is in control. Instead of writing a protocol as a script that imports and controls the robot, you write a protocol as a file defining a function that runs the protocol:
def run(protocol: opentrons.protocol_api.ProtocolContext):
# your code goes here
When you run or simulate a protocol, the Opentrons software stack parses the protocol and executes the function you have defined. The
protocol object, rather than being a representation of the robot, is a representation of the protocol: it is called the
ProtocolContext . Rather than always and forever existing and being shared by the robot server and your protocol,
ProtocolContext are created for every protocol run and destroyed afterwards. The robot server uses other lower level interfaces to control the robot. That means we can focus our efforts in development of the Protocol API V2 only on its use by scientists writing protocols, rather than having to compromise to support our own internal development.
All the interfaces to Protocol APIv2 are typed using Python type annotations, letting us catch common errors faster and providing you with more support when writing protocols. If you install the
opentrons module using
pip (as detailed in this article) the programs you use to write your protocols, like Visual Studio Code or Atom can provide more in depth autocomplete and help suggestions.
Protocol APIv2 has a much simpler system for determining positions, making problems with protocols or with robot positioning much easier to debug.
We're incredibly excited to share Protocol API v2 with you, and to use it to add even more incredible features and massively increase the reliability of the OT-2.
Please, opt in to the beta and give it a try!