FluenceArt: Transforming your images with MVVM and ESAPI
- mschmidt33
- Mar 19
- 12 min read
Preface
Looking to download FluenceArt? You can find it here on the Gateway Scripts website for download or on GitHub open-source.
Introduction
FluenceArt is a fresh take on an enduring concept where art meets radiotherapy. For years, creative developers and physicists have utilized applications to convert images to fluence maps, most famously the photo of Albert Einstein used to highlight the precision of multi-leaf collimator (MLC) leaf positioning. These novelty projects not only showcased the technical achievements of multi-leaf collimators, but also added a creative way to reproduce personal images. My first introduction to a tool of this nature was a tool built by a talented physicist and developer in the curriculum development space at Varian Medical Systems, who has since moved to Proton training. This application could accept an image and build an optimal fluence file -- a file allowing for the import of fluences generated with software outside of the primary treatment planning system (TPS).
FluenceArt builds on this legacy by harnessing modern development practices. Drawing on lessons learned from previous experiences and earlier blog posts on MVVM application design, this application is designed with clean, maintainable code at its core. The app is stylized utilizing a previous approach in a recently updated blog post regarding theme-based stylizing application user interfaces with AI assistance. For this particular UI, I requested the following from ChatGPT 4o:
Can you provide me with a color pallete that has a light background but the colors are more based in reds and oranges to simulate how fluence maps are shown in the treatment planning system?
In this blog post, FluenceArt will be introduced, and the development pattern and highlighted methods will be discussed. Since the application is available on open-source, there's no need to go through every line of code in the blog post. Instead we can focus on the interesting parts of the application, both in the UI design and image handling. In the final section of this post, practical guidance on installation and utilization of the application will be provided, complete with helpful tips to enhance your experience. Whether you're a seasoned medical physicist or simply intrigued by the intersection of art and science, I hope you find this exploration inspiring and innovative.
Design and Architecture
UI Design
In building FluenceArt, the visual experience was a top priority. The same GPT session from a prior blog post on stylizing applications using AI assistance was utilized to request an app that mirrored heatmap colors as if they were utilized in a fluence map in a radiotherapy treatment planning system. The resultant color scheme is as follows:
<!-- Light Theme Color Palette with Fluence Map Aesthetic -->
<Color x:Key="PrimaryColor">#FFFFF3E0</Color>
<!-- Soft warm background (off-white with a hint of orange) -->
<Color x:Key="SecondaryColor">#FFFF9800</Color>
<!-- Vivid orange for secondary elements -->
<Color x:Key="AccentColor">#FFD84315</Color>
<!-- Deep red-orange for accents -->
<Color x:Key="TextColor">#FF3D2C1E</Color>
<!-- Dark reddish-brown for readable text -->
<Color x:Key="BackgroundColor">#FFFFFBF0</Color>
<!-- Very light cream for main backgrounds -->
<Color x:Key="HighlightColor">#FFFFC107</Color>
<!-- Bright yellow for highlights -->
<Color x:Key="ButtonColor">#FFFF5722</Color>
<!-- Intense orange-red for buttons -->
<Color x:Key="SuccessColor">#FF66BB6A</Color>
<!-- Green for success messages, for contrast -->
<Color x:Key="ErrorColor">#FFD32F2F</Color>
<!-- Strong red for errors -->
<!-- Brushes for the Color Palette -->
<SolidColorBrush x:Key="PrimaryBrush" Color="{StaticResource PrimaryColor}"/>
<SolidColorBrush x:Key="SecondaryBrush" Color="{StaticResource SecondaryColor}"/>
<SolidColorBrush x:Key="AccentBrush" Color="{StaticResource AccentColor}"/>
<SolidColorBrush x:Key="TextBrush" Color="{StaticResource TextColor}"/>
<SolidColorBrush x:Key="BackgroundBrush" Color="{StaticResource BackgroundColor}"/>
<SolidColorBrush x:Key="HighlightBrush" Color="{StaticResource HighlightColor}"/>
<SolidColorBrush x:Key="ButtonBrush" Color="{StaticResource ButtonColor}"/>
<SolidColorBrush x:Key="SuccessBrush" Color="{StaticResource SuccessColor}"/>
<SolidColorBrush x:Key="ErrorBrush" Color="{StaticResource ErrorColor}"/>This leads to our nice light yellow background and vibrant UI full of colors. There are two areas of interest I'd like to highlight about the UI. First, the Image is loaded in the center column. The image can be changed based on a RadioButton selection underneath the image control visualization.
<!-- Center Panel: Preview Area -->
<Grid Grid.Column="1" Margin="10">
<Grid.RowDefinitions>
<RowDefinition Height="*" />
<RowDefinition Height="Auto" />
</Grid.RowDefinitions>
<Border BorderBrush="Black" BorderThickness="1">
<Image Stretch="Uniform" Source="{Binding ImageSource}"/>
</Border>
<StackPanel Orientation="Horizontal" Grid.Row="1" HorizontalAlignment="Center" Margin="0,10,0,0">
<RadioButton Content="Original Image" IsChecked="{Binding OriginalSelected}" Margin="10,0" />
<RadioButton Content="Fluence Map" IsChecked="{Binding FluenceSelected}" Margin="10,0" />
</StackPanel>
</Grid>Second, prior to being able to push to ARIA, a patient and plan must be selected. This is handled inside of an expander, which with the styling from the style theme looks and behaves well.
<!-- Right Panel: Export and ARIA Integration -->
<StackPanel Grid.Column="2" Margin="10">
<Button Content="Export Fluence Map" Margin="0,0,0,10" Command="{Binding ExportToFileCommand}" />
<!-- Optional: Expandable panel for patient details -->
<Expander Header="Patient Details" IsExpanded="False" Margin="0,20,0,0">
<StackPanel>
<TextBox Margin="0,5,0,5"
ToolTip="Enter Patient ID"
Text="{Binding PatientId,
UpdateSourceTrigger=PropertyChanged}"/>
<Button Content="Open Patient" Command="{Binding OpenPatientCommand}" Margin="0,5,0,5"/>
<ComboBox Margin="0,5,0,5" ItemsSource="{Binding Plans}" SelectedItem="{Binding SelectedPlan}"
DisplayMemberPath="Description"/>
</StackPanel>
</Expander>
<Button Content="Push to ARIA" Margin="0,10,0,10" Command="{Binding PushToARIACommand}"/>
<TextBlock Text="{Binding PushMessage}" Foreground="Red" TextWrapping="Wrap"/>
</StackPanel>Image Handling
When the image is imported, an OpenFileDialog class file explorer, allows the user to select from a number of various image file types. Once the file has been selected by the user, a new BitmapImage object is created with the UriSource being set to the filepath of the selected file. The Bitmap is then used as an imput source to the image in the UI. From there, FileInfo can be extracted and shown to the user regarding the image.
private void OnImportImage(object obj)
{
OpenFileDialog openFileDialog = new OpenFileDialog
{
Filter = "Image Files (*.png;*.jpg;*.jpeg;*.bmp;*.gif;*.tif;*.tiff)|*.png;*.jpg;*.jpeg;*.bmp;*.gif;*.tif;*.tiff|All Files (*.*)|*.*"
};
if (openFileDialog.ShowDialog() == true)
{
string filePath = openFileDialog.FileName;
try
{
// Load the image into a BitmapImage.
BitmapImage bitmap = new BitmapImage();
bitmap.BeginInit();
bitmap.UriSource = new Uri(filePath);
bitmap.CacheOption = BitmapCacheOption.OnLoad; // ensures the file is closed after load
bitmap.EndInit();
bitmap.Freeze(); // makes it cross-thread accessible
ImageSource = bitmap;
_originalImage = bitmap;
// Retrieve file info and update details string.
FileInfo fileInfo = new FileInfo(filePath);
ImageDetails = $"Filename: {fileInfo.Name}\n" +
$"Dimensions: {bitmap.PixelWidth} x {bitmap.PixelHeight}\n" +
$"Size: {fileInfo.Length / 1024} KB";
if(bitmap.PixelWidth>80 || bitmap.PixelHeight > 80)
{
ImageDetails += $"\nImage dimensions too large.\nImage will be downsampled automatically.";
}
FluenceModel fluenceModel = FluenceConverter.ConvertImageToFluenceMap(bitmap);
fluence = fluenceModel.Fluence;
origin = fluenceModel.Origin;
_fluenceImage = FluenceConverter.ConvertFluenceMapToHeatMap();
ExportToFileCommand.RaiseCanExecuteChanged();
if (SelectedPlan == null)
{
PushMessage = "Select a plan to push to ARIA";
}
}
catch (Exception ex)
{
// Update the details string with error information.
ImageDetails = $"Error loading image:
{ex.Message}\n{ex.InnerException}";
}
}
}After the image has been loaded from the file, two other methods are called. Once that converts the image into a fluence array, and another that converts that fluence array back into a heat map. The latter is only useful for showing the image in the UI as a heat map. The first method, ConvertImageToFluenceMap is the most important method in our FluenceConverter class. Below 2 methods are shared. The first task of the ConvertImageToFluenceMap method is to check the size of the image, and downsample the image if its too large.
At a fixed resolution of 2.5mm for fluence images, the image cannot be too large (in pixels) before it is outside the bounds of what the MLC can deliver. For example, Millennium MLC fields are 40cm in the Y-direction (160 pixels) whereas HDMLC are only 22cm (88 pixels). Moreover, MLC field width should be limited. If the fluence map is greater than 15cm (60 pixels) the field will split into multiple carriage segments, and if the field is greater than 30cm (120 pixels) the leaf motion calculator will consider the fluence map too large to convert.
At a fixed resolution of 2.5mm for fluence images, the image cannot be too large (in pixels) before it is outside the bounds of what the MLC can deliver.
The DownsampleIfTooLarge method will determine a transform factor that brings the largest dimension to 100pixels (25cm) to attempt to limit the height and width of the fluence map. After the image is appropriately downsampled, the image is converted into GrayScale (Gray8 image), and the bytes are extracted to build a new fluence. The fluence is then normalized to a maximum value of 1.0 as I considered that generally high enough to create IMRT fields of appropriate MU. It may also be worth noting how the Origin is calculated for the fluence map based on the size of the image as well. Empirically determined by reviewing fluence maps within the TPS, the origin of the fluence is negative in the X direction and positive in the Y direction for fluences centered on the field central axis.
public static FluenceModel ConvertImageToFluenceMap(BitmapSource source)
{
if (source == null)
throw new ArgumentNullException(nameof(source));
LocalFluence = new FluenceModel();
// Ensure the image is in a grayscale format (Gray8) for a single intensity value per pixel.
BitmapSource graySource = DownsampleIfTooLarge(source);
if (source.Format != PixelFormats.Gray8)
{
graySource = new FormatConvertedBitmap(graySource, PixelFormats.Gray8, null, 0);
}
int width = graySource.PixelWidth;
int height = graySource.PixelHeight;
// Allocate an array to receive pixel data.
byte[] pixels = new byte[width * height];
graySource.CopyPixels(pixels, width, 0);
// Determine the maximum pixel value to use for normalization.
byte maxPixel = 0;
foreach (byte pixel in pixels)
{
if (pixel > maxPixel)
maxPixel = pixel;
}
// Prevent division by zero.
if (maxPixel == 0)
maxPixel = 1;
// Create the fluence map as a 2D float array.
float[,] fluence = new float[height, width];
for (int y = 0; y < height; y++)
{
for (int x = 0; x < width; x++)
{
byte pixelValue = pixels[y * width + x];
// Normalize pixel value to the range [0, 1].
float normalizedValue = pixelValue / (float)maxPixel;
fluence[y, x] = normalizedValue;
}
}
// Define the resolution: 2.5mm per pixel = 0.25cm per pixel.
float resolution = 2.5f; // in mm
// Calculate the origin position
float originX = -((width - 1) / 2.0f) * resolution;
float originY = ((height - 1) / 2.0f) * resolution;
Point origin = new Point(originX, originY);
//set local fluence
LocalFluence.Fluence = fluence;
LocalFluence.Origin = origin;
return new FluenceModel()
{
Fluence = fluence,
Origin = origin
};
//return (fluenceMap, new Point(originX, originY));
}
public static BitmapSource DownsampleIfTooLarge(BitmapSource source, double samplingResolution_cm = 0.25, double maxPhysicalWidth_cm = 20.0)
{
if (source == null)
throw new ArgumentNullException(nameof(source));
int maxLength = Math.Max(source.PixelWidth, source.PixelHeight);
// Calculate the physical width in cm.
double physicalWidth_cm = maxLength * samplingResolution_cm;
// Check if the physical width exceeds the maximum allowed.
if (physicalWidth_cm > maxPhysicalWidth_cm)
{
// Downsample by applying a ScaleTransform with a scale factor of 0.5.
double scaleFactor = Math.Round(100.0 / maxLength,1);
var scaleTransform = new ScaleTransform(0.5, 0.5);
var transformedBitmap = new TransformedBitmap();
transformedBitmap.BeginInit();
transformedBitmap.Source = source;
transformedBitmap.Transform = scaleTransform;
transformedBitmap.EndInit();
transformedBitmap.Freeze(); // Make it cross-thread accessible.
return transformedBitmap;
}
return source;
}
ESAPI Integration
For seamless integration with Eclipse's treatment planning systems, FluenceArt utilizes ESAPI to import the fluence map after its derivation. An ESAPIAutomationService class handles all automation events within the application. With separate methods to SetPatient, SetCourse, SetPlan based on user provided input, the ESAPIAutomationService's primary method is SetFluence().
public static class EsapiAutomationService
{
public static ExternalPlanSetup Plan;
public static Patient Patient { get; private set; }
public static Course Course { get; private set; }
public static void SetPatient(Patient patient)
{
Patient = patient;
}
public static void SetCourse(Course course)
{
Course = course;
}
public static void SetPlan(PlanSetup plan)
{
Plan = plan as ExternalPlanSetup;
}
public static void BeginModifications()
{
Patient.BeginModifications();
}
public static bool SetFluence(bool autoCalculate, float[,] fluence, Point origin)
{
//get first field.
Beam beam1 = Plan.Beams.First(b => !b.IsSetupField);
//construct fluence.
Fluence fieldfluence = new Fluence(fluence, origin.X, origin.Y);
beam1.SetOptimalFluence(fieldfluence);
return true;
}
}The SetFluence method will find the first treatment field in a plan, and call the ESAPI SetOptimalFluence method. Once the method is returned, the script then calls SaveModifications on the Application object. This feature allows for the user to deploy the app outside of an ARIA environment and then still be able to import the optimal fluence map directly into Eclipse.
public void OnExportImage(object obj)
{
float spacingX = 2.5f;
float spacingY = 2.5f;
if (fluence == null)
throw new ArgumentNullException(nameof(fluence));
int height = fluence.GetLength(0);
int width = fluence.GetLength(1);
SaveFileDialog saveFileDialog = new SaveFileDialog();
saveFileDialog.Filter = "Optimal fluence file (*.optimal_fluence)|*.optimal_fluence";
saveFileDialog.RestoreDirectory = true;
saveFileDialog.Title = "Save fluence file";
if (saveFileDialog.ShowDialog() == true)
{
// Create or overwrite the output file.
using (StreamWriter writer = new StreamWriter(saveFileDialog.FileName))
{
// Write header lines.
writer.WriteLine("# Field 1 - Fluence");
writer.WriteLine("optimalfluence");
writer.WriteLine($"sizex\t{width}");
writer.WriteLine($"sizey\t{height}");
writer.WriteLine($"spacingx\t{spacingX.ToString(CultureInfo.InvariantCulture)}");
writer.WriteLine($"spacingy\t{spacingY.ToString(CultureInfo.InvariantCulture)}");
writer.WriteLine($"originx\t{origin.X.ToString("F4", CultureInfo.InvariantCulture)}");
writer.WriteLine($"originy\t{origin.Y.ToString("F4", CultureInfo.InvariantCulture)}");
writer.WriteLine("values");
// Write the fluence values row by row.
for (int y = 0; y < height; y++)
{
string[] rowValues = new string[width];
for (int x = 0; x < width; x++)
{
// Format each value with a suitable numeric format.
rowValues[x] = fluence[y, x].ToString("G6", CultureInfo.InvariantCulture);
}
writer.WriteLine(string.Join("\t", rowValues));
}
}
}
}Fluence Art User Guide
Installation
Fluence Art can be installed for use via two mechanisms: open-source code or download.
Download:
The pre-compiled executable files have been made available on the Gateway Scripts website at the GatewayCode link. From this site, click Download on the Fluence Art link.
After completing a short form and agreeing to an agreement statement, one of three links can be used to download the code. The downloaded code will be compressed into a zip folder. It may be worth to check in the zip folder properties if the files within can be "Unblocked". You may extract the files from that folder and use within an Eclipse environment or on your own workstation per the section titled Online vs Offline Mode.
Source Code:
Alternatively, for those more developer savvy, the source-code can be cloned directly from the Fluence Frame repository on Github. After cloning the code to your flavor of Visual Studio, you may want to select a branch most closely associated with your version of ESAPI from the Git Changes tab in Visual Studio. It may also be required to re-attach the ESAPI references (VMS.TPS.Common.Model.API and VMS.TPS.Common.Model.Types) if they are not automatically discovered on build.
UI Overview
Each control is designed to guide you through the process- from importing an image and visualizing the fluence map to exporting data or integrating the export with ESAPI - making the workflow both intuitive and efficient. Below is a short guide to the buttons and controls in FluenceArt's UI:
Import Image: Opens a file explorer so you can select an image file. Once selected, the image is loaded into the application, its details (such as filename, dimensions, and size) are displayed, and the processing to generate the fluence map begins.
Radio Buttons (Original Image/ Fluence Map): Located below the image preview, these allow you to toggle between viewing the original imported image and its corresponding fluence map (displayed as a heat map).
Export Fluence Map: Saves the generated fluence to a text file in the required format. This file includes header details (dimensions, spacing, origin) and the normalized fluence values, ready for use in the treatment planning system or further analysis.
Patient Details Expander: Not a button per se, but an interactive panel where you can enter the Patient ID, open the patient record, and select the appropriate plan. This step is essential before pushing the fluence map to ARIA.
Please note: The plans are listed in the format "Plan Id [Course Id]" to differentiate between plans of equal identifiers in different courses.
Open Patient: Within the Patient Details panel, this button opens the patient record in ARIA, ensuring that the right patient context is set.
Push to ARIA: After the patient and plan have been selected, this button transfers the generated fluence map into ARIA automatically by setting the optimal fluence for the selected treatment field.
Online Vs. Offline Mode
The application itself can be run in an online mode (within an Eclipse environment) or offline (without Eclipse). In an Eclipse environment, the patient information can be selected in the expander, and Push to ARIA can push the fluence map to the first beam of your selected plan.
If there is a desire to run in offline mode, the user must first open the file FluenceFrame.exe.config - this file can be opened by a simple text editor such as NotePad. Find the app setting with the key "RunMode". The value should most likely be "ESAPI". If you change the value to "NoESAPI" the application will not attempt to connect to the Eclipse Scripting API, and the application can be run on any computer or workstation. In this mode, the only option will be to export the fluence map to a file, and import into Eclipse at another time.
The Eclipse Fluence Editor is a valuable tool in the event that the fluence is too large for the leaf motion calculator or perhaps you'd like to remove some background items leading to large fluence values.
One the application is run, and the optimal fluence imported into Eclipse, it may be possible to further edit the fluence by right mouse clicking the fluence, and selecting the option "Edit Fluence". The Eclipse Fluence Editor is a valuable tool in the event that the fluence is too large for the leaf motion calculator or perhaps you'd like to remove some background items leading to large fluence values. Finally, the dose can be calculated. Calculate the dose, and ensure that the MU is enough for the delivery type. For example, you may need to multiply the MU by a factor of 5-10 if delivering to film.
If delivering to film, remember that adding some buildup can help the image increase signal relative to the amount of MU delivered. In this case, I decided to deliver to an aS1200 DMI Electronic Portal Imaging Device. It was surprising to see the image construct itself in on the imaging application. The detail is astonishing!
Conclusion
In closing, FluenceArt represents an exciting convergence of art and advanced radiotherapy technology. This project builds on a rich tradition of creative imaging techniques while harnessing modern development practices such as MVVM and ESAPI integration. By transforming everyday images into sophisticated fluence maps, FluenceArt not only offers a novel tool for treatment planning but also opens the door to a unique blend of technical precision and artistic expression.
FluenceArt's detailed UI design, intuitive controls, and robust image processing workflow ensure that both clinical professionals and tech enthusiasts can navigate the application with ease. Whether you're running the app online within an Eclipse environment or in standalone offline mode, the flexibility and functionality of FluenceArt make it a valuable tool for transforming your photos into a new medium.
I invite you to try out FluenceArt, available on GatewayScripts.com, and share your experiences and feedback. Your insights will be crucial in shaping future enhancements and expanding the capabilities of this innovative tool.