TPW Workflow Series: Processing 3D scans
This blog of the TPW workflow series describes how TPW processes 3D scans to a 3D object (mesh). There are many types and brands of 3D scanners, varying from laser to structured light (SLS), handheld and stand-alone. This blog describes the procedure of processing scans with a stand-alone, SLS machine, the DAVID SLS-3.
This scanner is set up in a fixed position and does not move during scanning. Therefore, the targeted object needs to rotate, enabling the scanner to record every part of the object’s surface. This produces multiple scans (pointclouds). The software is theoretically able to automatically stitch (‘align’) the separate scans together into a whole, but due to the morphological complexity of archaeological pottery, specifically sherds, the automatic alignment fails frequently and the operator needs to manually align the scans. This process of alignment will be described in this workflow. After successful alignment, the scans can be fused to a polygon mesh (or just mesh): all the points that make up a scan are connected as to form a net, called a wireframe. This collection of vertices, edges and faces describe the shape of the 3D object. A vertex is a point, the edge connects those points and a face is the surface enclosed by the edges.
Because the scans were made with the highest resolution possible, the meshed outputs have an incredibly large filesize, often surpassing 1GB. In order to actually use the scans on regular computers, the meshes should be simplified, also called decimation. Subsequently, the decimated meshes may be exported to other filetypes in order to use them in different software packages and for online sharing. This ‘post-processing’ phase is treated in the TPW workflow Post-processing 3D models.
Finally, it is imperative to document the path that lead to the creation of a workable 3D object, and to record what has been changed compared to the original scan sequence. Also choices in decimation algorithms or what to cut away from the scans are important to document. The TPW workflow Democratising data is committed to this important process of keeping so-called meta- and paradata records.
The 3D scans are made almost exclusively with the DAVID-SLS3 (now called HP 3D Structured Light Scanner Pro S3), and this manual is dedicated to this particular 3D scanner and software.
Processing 3D scans
Step 1: Which file to choose
DAVID produces HP 3D Scan Project files: *hp3dscanproj. These project files can only be accessed and worked with when the hardware-key (memory stick with software license) is plugged in.
Open the *hp3dscanproj and save the project directly as ‘cut’ or with your initials: for example
‘11381_NativeFile’ or ‘11381processed’.
You can find the ‘save project as’ button in the right upper corner (a little disk with a red capital A and a small red a). In doing this, you always maintain a copy with the original scans.
Step 2: Getting started
When you opened the project file, you will see a bright colored ceramic vessel. All different colors refer to separate scans. To the left of this view port with the colored vessel, is a window with tools. From up to down these tools are : ‘Cleaning’, ‘Alignment’, ‘Fusion’ and ‘Comparison’. The manual is mainly concerned to the first three tools.
To the right of the view port is the ‘List of Scans’ and above that ‘Project’, with the name of the project and options to delete (the red X), open/load projects (folder with green arrow) and two save as and save buttons. If the project name shows an asterix to the right, then the project needs to be saved: press the save-button to the far right.
Interacting with the 3D object
When clicking with the right mouse button on the 3D object you can rotate the object. You also see a trackball (sphere). The object should fit exactly in the middle of the trackball, otherwise rotating the object will be off centered, which is extremely annoying. To move the object into the sphere, just use theleft mouse button. To zoom on the object, use the mouse-wheel.
Above the view port 7 icons. To the left is an eye, this adjusts the view back so that all separate scans are viewed at once. To the right of the eye is an icon displaying a little picture frame (with a sun).
Show textures (T)
This button enables to display the texture (photo layer); or shortcut ‘T’. We usually work in the non-textured mode.
To undo something, press ctrl+z or hit the left green circular arrow button in the Alignment-tab. To redo something press ctrl+y or the right green circular arrow button.
In the List of Scans there is an icon of an eye. The icon above the List is to turn all scans on or off. With the eye-icons in the list you can turn on and off the scans separately. Next to the eyes in the List of Scans is a colored square. You can check and uncheck these squares (i.e. scans).
When a square is checked, a blue box will appear around the designated scan in the 3D view port.
To move one individual scan outside a collection of scans, select the particular scan by checking the box in the List of Scans. Then hold shift while dragging the scan with the left mouse button.
Under the List of Scans there is a + and a -. With the + you can import scans. This sounds better than it actually is: you cannot import native scans, only meshes such as obj. With the – button you can remove scans from the list (when checked, see above). To the right is an icon of disk with red Aa, to export separate scans or fused results to obj.
Below the view port is an information bar. When aligning or fusing, a progress bar will appear. This means nothing and certainly does not reflect actual running or remaining time. The cancel button does not make much of a difference either.
If you wish to see the scans as separate individuals instead as a whole, go the Alignment tab. Under ‘Alignment’ are two buttons: one with 6 squares in two rows and one with 3 squares. These buttons arrange the separate scans next to each other.
Step 3 Cleaning the scans
At the point where the structured light beams hits the surface of an object and crosses with the depth-range of the camera, which records that particular lit area on the surface, the best accuracy is reached. The further away from that point, towards the edges of the scan, the more blurred and distorted the scan is. In order to reach the best alignment (= precise overlap between the scans), resolution and to avoid ridges of overlapping scans in the final complete object, these bad parts need to be cut away.
Select the scan that needs cleaning by pressing the eye-button in the List of Scans (turn off all scans), and turn (the eye) on of the preferred scan. Go to Cleaning and press the grey button with the wind shield-like icon (or press S); this will turn blue.
Press on the drop-down menu next to it and choose ‘Polygon Selection’.
Now click around the parts you wish to remove from the scan and double-click when finished. Now the selected part will turn red. Press the red X in the Cleaning-tab or hit DEL. The bad part of the scanned is now removed. You cannot pan around the object when the cleaning tool is on; you can only rotate with the right mouse button.
Repeat this step with each scan. BUT be careful, regularly check if you don’t cut away too much of the scan. ALWAYS check every now and then all the scans to check if you are not creating holes in the complete 3D object.
Be aware that when a selection has been made on one scan and after that the other scans are made visible again and you press DEL, that selected part will be deleted in all scans, creating a huge gap. When deleting, ONLY turn on/visualise the specific scan.
Pay attention to sharp edges. The edges of scans are sometimes ‘folded’, leaving a ridge when overlapping/merging the scans. Cut these ridges away. Some edges have ‘shark-teeth’ – triangular projecting arrows that stick out for nor reason. These are obviously not original: cut these away.
Step 4 Alignment
Alignment is the actual stitching together of the separate scans. The software is technically able to perform this automatically, but usually this fails when dealing with complex objects such as archaeological ceramics.
In the Alignment tab is a drop-down menu. Here are four methods of alignment: Free, Around Axis, Pairwise Fine Registration and Global Fine Registration. The most common used are Free and Pairwise Fine Registration. These two methods are described below.
In Free there are two options: manual or automatic alignment. Start with automatic alignment. If this goes bad, then try the manual way.
Check the ‘Use Texture’ (if there is a texture) and ‘Use Surface Features’ boxes. Then press the ‘Align Scans’ (G) button with the little red magnet icon. Two fields with ‘Scan A’ and ‘Scan B’ will appear. Scan A will be the scan that needs to be aligned with the first one. So, in the List of Scans, select for example Scan 2 (this will appear in the Scan A field) and then Scan 1 (this will appear in Scan B). These scans will merge automatically.
Note that, in case of for example 12 scans, that you work in two directions: first start with aligning Scan 12 to Scan 1, and work backwards. 2>1, 3>2, 4>3, 5>4, 6>5 and then 11>12, 10>11, 9>10, 8>9, 7>8, but the numbers in the middle are tricky: see what fits best. Scan 12 is really close to Scan 1 because one scanning round of 360 degrees means that Scan 1 at the start is, say, at 0 degrees and Scan 12 at the end at 360 degrees. They therefore need to be aligned best.
In the ‘Free’ environment, check ‘Contact Pair Selection’. The ‘Use Texture’ box will be automatically unchecked, and cannot be checked when there is opted for manual alignment. A red circle and little magnet appear now at the cursor. Go to the desired two scans. Separate the two to be aligned scans sufficiently (for dragging scans see Step 2) to create enough overview. The texture layer can be of great assistance now. Click on a feature that is present on both scans. A green dot with red line appears. Then click on that same feature on the other scan. Now the scans will be merged. If the this does not happen automatically and the line stays green: double click.
Step 5 Fusion
When the scans are perfectly aligned, the project can be fused into one, solid whole: a mesh. Go to the ‘Fusion’ tab. Here are three options: ‘Resolution’, ‘Sharpness’, and ‘Close Holes’.
Resolution: Ideally we would like the best result and therefore would like to move the scale to its maximum of ‘4000’, creating a vertex spacing (distance between points) up till 40 microns (0,04xx). But somehow this is not possible. Next to the fact it takes ages (12-20 minutes of calculation time) to process, it creates transparent parts in the mesh. This is an issue that still needs to be resolved. For now, move the scale to 2000. If this creates a bad result too, move the scale to less than 2000.
Sharpness: leave it to 0. When ridges between overlapping scans appear, you could consider to move the sale to -1 (but not more).
Close holes: set the scale to 1%. Increasing the value create strange blobs.
Now press the Fuse button, with the little green Earth icon on it.
After Fusion, add the Fusion Result to the List of Scans. Go right to the view port where you will find the Fusion Result tab. If the result is bad, choose the red X to delete, if it is OK, choose the middle button with the green arrow and little notepad. Compare the FusionResult with the scans by checking and unchecking the eyes/scans: are all the features visible? Save the project.
Step 6 Export
Is the FusionResults OK? Then check the box of the FusionResult and press the Export button in the right left corner under the List of Scans. Save it as an *OBJ.
Problems and troubleshooting
DAVID is an extremely sensitive little machine. It records everything. Therefore, a solid environment is needed, but this is not always the case. For example: our department is situated in the attic of an old building. When the train passes by or if it is stormy weather, the building is shaking. DAVID records these vibrations, and turns this into unnatural (horizontal/oblique/vertical) parallel running lines in the scans. Ignore these lines, unless you have an overlapping with little or no lines, then remove the part with the lines.
Black absorbs light, so where there is black on the surface of an object, such as an inventory number written in black ink, or black painted decoration, this cannot be recorded by DAVID and leaves a hole. The software will fill this hole. There is not much you can do about this at this point. Be aware though that this is not an original feature.
Next step: post-processing in Meshlab
The post-processing phase will take place in open source software Meshlab. Here we will simplify, or decimate the meshes to more workable (i.e. smaller) files, which can be used by other users with for example limited computing power or who would like to download and analyse the TPW models in particular software.
The post-processing workflow is described in the TPW workflow episode Post-processing.