By: Liron Shimrony
Intro
A while ago I was asked to build an interface for a medical volumetric segmentation algorithm[1][2]. The goal of this project was to create an interface where specialists would be able to calculate body organs' volumes or cancer tumors' volumes by very crude brush strokes, where blue represents a non-target pixel and red represent a target pixel.
In order to create the 3 views (axial, coronal and sagittal), I needed to parse the Dicom files (medical imaging format), build a 3D cube, slice it in different directions and save each slice to the server. In order to move between slices, the user needs to scroll up or down when the mouse cursor is on the requested view.
One of the most important features is to have the ability to send to our backend a list (or array) of pixels which represents the original image and another two list of pixels which represents the brush strokes:
- Target pixels (red)
- Non-target pixles (blue)
I originally built this web application using Django framework but I decided it would be an educational exercise to recode it in Javascript. My choice was Express.js, Node.js and Angular.js. I also decided that I want to display the images using SVG components so I can take advantage on their transformations (such as scale, rotate, etc.). In the original web application, the images were in a PNG format so those transformations were harder to be achieved. The question that I asked myself was what is the best way to store the images on the server?
Notes:
- For dicom parsing I used dicomParser library.
- For the purpose of this article I will use the following 5 images which show a human brain from a top (axial) view
Trial 1- Saving Dicoms Files as Plain SVG
My naive approach was to create a line with a length of one pixel and assign it the current pixel color. This is a partial code of the generated SVG:
I ended up with a text file which has a size of 2.6MB. If we calculate the amount of storage needed to store the above study which contains 176 slices we will get
2.6MB * 176 slices = 457.6MB only for the axial view. I assume that the other two views (sagittal and coronal) would cost another 600MB (276 slices for both) and we will end up with a modality which takes 1GB of storage. The above web application can show multiple modalities at once. Considering a study which contains two modalities, we will need to load 2GB of data which is not feasible. The process of loading the whole 176 slices and convert them to SVG files took about 3 minutes which is very long time comparing to about 10 seconds in the web application which is written in Python.
Trial 2- Converting Dicoms to SVG Path
After spending some more time thinking about a solution, I decided on another approach- using SVG path. Instead of storing each pixel location and color, we iterate over all the pixels and create a line from point A to point B only when there is a change in color. The results are being saved to a text file in json format since SVG has a lot of redundant text.
The following is a partial text file that was generated using the method above:
HTML EJS template:
The result SVG of the above text file:
This way, I managed to reduce the file size for each image to about 800KB. So for the study above, we need 800KB * 176 slices = 137.5MB for the axial view, which is a huge improvement comparing to the first method but still not good enough. However, comparing to the first method, parsing the dicom file and saving it as text, took about 1 minute.
Trial 3- Storing Hex Values
After thinking about the problem a little more I came up with a new approach: saving an array of hex values which represents the color of every pixel. This way, I can get the width and length of the image and assign the right color for every pixel.
Text file example:
HTML EJS template:
This way, the file size of each image was reduced to 346KB which is a total of 346KB * 176 slices = 59.47MB, about 16% from our first trial. Also, parsing the dicom data and saving it as text took less than a minute. I was very happy with my achievement but then I found out that iterating over the pixels of the above images (in order to display the images to the user) takes longer than I thought. I decided to check the performance of my algorithm on a larger image (512 * 512 pixels) and the result was awful. It was way too slow to work with. I had to find a completely different approach.
Trial 4- Let's Try Something New
I guess that there is no quick way to iterate over a pixel array, especially when the image is big. Also, after a little bit more research, I found out that we can use an <image> tag inside an <svg> element and still enjoy the scaling and zooming of SVG. The next thing that came to my mind is converting each image to Data URI and save it as text.
For loading dicom file as an image I used cornerstone library.
Example
Result:
Getting the Pixels:
ThecornerstoneFileImageLoader parses the dicom file and places it as an image inside a <canvas> element, which makes it easier to access its pixel data:
Converting the canvas into Data URI:
Using this method, each image, when saved as text takes about 12KB, which is a total of 12KB * 176 slices = 2.11MB. Nevertheless, uploading and parsing a series of images takes a few seconds. Last but not least, it is very easy to send the pixels' data to the server as an argument if a function needs to use it using the method above.Conclusion
SVG elements are quite powerful and can make our job lot easier. However, rendering the SVG elements can take some time depending on the size of our data. Using SVG elements for a series of images might not be optimal. However, by doing so, we can take advantage of their great functionality such as scaling, rotating, translating, etc.
As for sending the user's brush strokes data to the server, I just created 2 SVG elements one on top of the other. The bottom SVG contains the original image while the top one contains circles which represents the user's decision (blue for non-target pixels and red for target pixels). The two types of brush strokes have a different class. Thus it is easy to differentiate between them when inspecting the data on the server side.
For more information about SVG please visit MDN
Notes
- Segmentation algorithm was written by
- Michael Grossberg - Assistant Professor at CUNY City College of New York.
- Gig Mageras - Memorial Sloan Kettering Cancer Center
- Yu-Chi Hu - Memorial Sloan Kettering Cancer Center.
- You can find more information about the algorithm at here.
About the Author
My name is Liron Shimrony and I am a web developer with a strong passion for technology. I earned my B.Sc in Computer Science in the City College of New York and currently working towards my M.S in Computer Science.If you have any questions, please contact me via my website.
