Imaris Open - Discussion Forum

This is a community discussion space where Imaris is the key topic. This forum will offer guidance to new users, technical as well as application support. Users from sites with a valid Imaris Infinity subscription will receive priority – however, the fastest way to get to the Bitplane Technical and Application Support team will continue to be via Asia, EU or US support mailboxes. Every registered user can add items to the forum. Non-registered users can read the forum but cannot add/comment, etc. All ideas, questions, suggestions and comments are welcome.

Advanced Search
Thread Author
Max. Age of Thread
Number of Posts
Thread Rating
HomeHomeXTensionsXTensionsPythonPythonFastest way to transfer large datasets from Imaris to Python?Fastest way to transfer large datasets from Imaris to Python?
New Post
Fastest way to transfer large datasets from Imaris to Python?
 

Hi everyone,

I wanted to compare notes with anyone trying to resolve this.

Edit: I've now tested Imaris 9.2.0 put all my findings in a Jupyter Notebook:

https ://nbviewer.jupyter.org/github/zindy/Imaris/blob/master/tutorials/bridgelib_datavolume.ipynb

The stack I'm playing with is 2200x2688x131 (16 bit). That's almost 1.5GB of data / channel / timepoint.

Transferring a 16 bit dataset from Imaris -> Python is still slow, but transferring from Python -> Imaris is now OK for both 8 bit and 16 bit datasets (twice the time it takes to transfer an 8 bit dataset, so can't complain).

What I still can't do without Imaris (or Python) trying to allocate more memory than I have is:

    arr = vDataSet.GetDataVolumeAs1DArrayShorts(0,0)

When the operation eventually succeeds, it takes about 226s, at some point rendering Windows completely unresponsive. Transferring data slice by slice is the way to go for now (takes half the time and Windows remains responsive throughout).

So this is the code I use in my BridgeLib library (updated, no need to swap axes)
:

    nx = vDataSet.GetSizeX()
    ny = vDataSet.GetSizeY()
    nz = vDataSet.GetSizeZ()
    dtype = GetType(vDataSet)

    if dtype == np.uint8:
        s = vDataSet.GetDataVolumeAs1DArrayBytes(aIndexC,aIndexT)
        arr = np.frombuffer(s,dtype).reshape((nz,ny,nx)).copy()
    else:
        #We define an empty array of the final size
        arr = np.empty(nz*ny*nx,dtype)

        if dtype == np.uint16:
            GetData = vDataSet.GetDataSubVolumeAs1DArrayShorts
        elif dtype == np.float32:
            GetData = vDataSet.GetDataSubVolumeAs1DArrayFloats

        #Filling-up the array
        for z in range(nz):
            arr[z*ny*nx:(z+1)*ny*nx] = GetData(0,0,z,aIndexC,aIndexT,nx,ny,1)

        arr = arr.reshape(nz,ny,nx)

    return np.ascontiguousarray(arr)


Pushing a volume from Python to Imaris is much faster. This is the code I use (updated, no need to swap axes):

    nx = vDataSet.GetSizeX()
    ny = vDataSet.GetSizeY()
    nz = vDataSet.GetSizeZ()
    dtype = GetType(vDataSet)

    #Make sure the data is in range and convert the array
    s = arr
    if dtype != arr.dtype:
        miset,maset = GetRange(vDataSet)
        arr[arr         arr[arr>maset]=maset
        s = arr.astype(dtype)

    if dtype == np.uint8:
        SetData = vDataSet.SetDataVolumeAs1DArrayBytes
        s = s.tostring()
    elif dtype == np.uint16:
        SetData = vDataSet.SetDataVolumeAs1DArrayShorts
        s = np.ravel(s)
    elif dtype == np.float32:
        SetData = vDataSet.SetDataVolumeAs1DArrayFloats
        s = np.ravel(s)
    SetData(s,aIndexC,aIndexT)


UPDATE (new code above, Imaris 9.2.0) This only takes 9 seconds for a 16 bit array (including type conversion) and 5 seconds for an 8 bit array (again including type conversion).

Question still remains: Any way to make this even faster? And what about Imaris -> Python for 16 bit data? Between numpy arrays, python lists, GetDataVolume / GetDataSubVolume / GetDataSlice +/- As1DArray, has anyone hit a on a sweet spot combo to quickly transfer large volumes from Imaris into Python?

Cheers!

 
Copy Post URL


Post URL:
casque beats pas cher casque beats by dre pas cher beats by dre pas cher beats by dre pas cher maillot foot pas cher maillot de foot pas cher maillot foot pas cher 2014 maillot de foot pas cher maillot foot pas cher maillot de foot pas cher maillot foot pas cher 2014 maillot de foot pas cher maillot foot pas cher 2014 maillot de foot pas chere maillot foot pas cher maillot de foot pas cher Ralph Lauren Homme Polo Manche Longue Ralph Lauren Polo Ralph Lauren Pas Cher Homme Polo Ralph Lauren Homme Pas Cher Ralph Lauren Soldes Ralph Lauren Big Pony Ralph Lauren Soldes Ralph Lauren Big Pony Ralph Lauren Soldes Ralph Lauren Big Pony Ralph Lauren Soldes Ralph Lauren Big Pony