Semantic Segmentation vs. Instance Segmentation: What’s the Right Workflow for Your Data?
July 28, 2026
Image segmentation is a crucial step in many scientific and industrial workflows, and a core capability of Dragonfly 3D World. Whether you’re analyzing biological samples or identifying defects in a manufactured component, accurate segmentation lays the foundation for meaningful analysis and measurement at a later stage.
As datasets grow larger and more complex, manual segmentation becomes increasingly time-consuming and difficult to reproduce. AI-powered tools help address this challenge by making segmentation more consistent, easier to scale, and often significantly faster than manual workflows. Dragonfly 3D World supports both semantic and instance segmentation workflows through tools such as the Segmentation Wizard and Cellpose-SAM, giving researchers and industry professionals the flexibility to choose the right approach for their data.
Not All Image Segmentation Workflows Are Created Equal
Within the field of image processing, there are two primary types of segmentation, each with their own purpose and workflow.
Semantic segmentation is primarily about classification, separating objects into different classes, and assigning all pixels belonging to the same category as a single label. For example, every void within a cast part or every mitochondrion within a cell dataset would be identified as part of the same class.
Instance segmentation goes a step further by separating each individual object into its own class. So, rather than identifying all mitochondria as one group, instance segmentation distinguishes each mitochondrion as a separate object. This is essential for counting and analyzing individual structures within a sample.
While both approaches can be used to segment data, they’re designed to answer different questions. Understanding those differences is the key to selecting the right workflow for your application.
Semantic Segmentation: Training Models for Repeatable Results
One way to think about the difference is as a bucket of popcorn. With semantic segmentation, the goal is to identify the popcorn as one class and the bucket as another. Every popcorn kernel belongs to the same category. Instance segmentation takes this a step further by identifying and separating every individual kernel.
Before Segmentation
After Segmentation
An easy semantic segmentation example, highlighting the salt and sugar crystals within a sample tube.
The same principle applies to scientific and industrial datasets. Imagine a metal casting containing internal voids. If your goal is to identify where porosity exists and determine how much of a material is occupied by voids, semantic segmentation is often the best choice. All voids are treated as part of the same class, allowing you to measure overall porosity across a sample.
However, if you need to count each individual void, measure its size, or group defects by diameter, instance segmentation becomes necessary since each void needs to be identified as a separate object.
When consistency across multiple datasets is important, semantic segmentation is often the preferred approach. Dragonfly’s Segmentation Wizard provides an intuitive workflow for training classical and deep learning models based on your data. Once trained on a representative region of interest (ROI), the model can be applied to an entire 3D dataset or reused across similar ones, helping reduce manual effort while maintaining reliable results.
For example, in our article on AI-enhanced casting porosity analysis, a deep learning model was trained to identify porosity within a CT-scanned casting. Rather than separating and measuring each individual pore, the goal was to classify all porosity regions and quantify defects across the sample. The trained model successfully identified subtle shrinkage pores and narrow voids that conventional thresholding methods missed. It also produced consistent results when applied to a different casting sample.
In short, semantic segmentation is often the best choice when you need to:
- Reduce manual segmentation time
- Separate different components/phases in your data
- Get an overview of material distribution
This makes semantic segmentation particularly effective for analyzing features across large and complex datasets.
Instance Segmentation: Separating Individual Objects with Cellpose-SAM
Semantic segmentation can tell you where porosity exists and how much of the material is occupied by voids. But if you need to know how many voids are present, which defects are the largest, or how pore sizes are distributed throughout the sample, each void must be identified as a separate object. This is when instance segmentation is the better choice.
Dragonfly's Cellpose-SAM combines the strengths of Cellpose with Meta's Segment Anything Model (SAM) to provide fast, accessible instance segmentation. In many cases, users can separate touching or overlapping objects with just a few clicks, without the need to train a custom model.
This workflow is particularly valuable when object-level measurements are required, such as counting cells or analyzing particle size distributions. By automatically separating structures into individual instances, Cellpose-SAM enables quantitative analysis that would not be possible if those structures were grouped together.
As a result, instance segmentation is often the better choice when you need to:
- Separate touching particles, pores, or cells
- Count individual objects
- Perform object-level quantitative analysis
- Explore a dataset quickly before deeper analysis
Choosing the Right Image Segmentation for Your Data
There’s no single "best" approach when comparing semantic segmentation vs. instance segmentation. It all depends on the type of analysis you need to perform.
If your goal is to identify and analyze a feature as a whole, semantic segmentation is often the right choice. If you need to analyze individual structures, instance segmentation is typically the better option.
In many workflows, the two approaches complement one another. Semantic segmentation can first identify a region or class of interest, while instance segmentation can then separate and analyze the individual objects within that class.
Together, Dragonfly's Segmentation Wizard and Cellpose-SAM help transform image segmentation from a manual bottleneck into a scalable workflow, enabling researchers and industry professionals to spend less time preparing data and more time generating insights.
Want to see these workflows in action? Explore Dragonfly 3D World's segmentation tools with a 30-day free trial and discover how AI-powered image segmentation can accelerate your next project.