Module package

Submodules

Module.Data_Input module

Module.Data_Input.check_data_type_format(dataType: str, dataFormat: str, logFile=None)

Check setting for dataType and dataFormat

Parameters:

• dataType – ‘Surface’, ‘Volume’, ‘Surface-Volume’

• dataFormat – ‘HCP Surface (.cifti, *.mat)’, ‘MGH Surface (.mgh)’, ‘MGZ Surface (.mgz)’, ‘Volume (.nii, .nii.gz, *.mat)’, ‘HCP Surface-Volume (.cifti)’, ‘HCP Volume (*.cifti)’, ‘FreeSurfer’

• logFile –

Yuncong Ma, 10/18/2023

Module.Data_Input.check_template_type_format(dataType: str, templateFormat: str, logFile=None)

Check setting for dataType and templateFormat

Parameters:

• dataType – ‘Surface’, ‘Volume’, ‘Surface-Volume’

• templateFormat – ‘HCP’, ‘FreeSurfer’, ‘3D Matrix’

• logFile –

Yuncong Ma, 10/10/2023

Module.Data_Input.compute_brain_surface(file_surfL: str, file_surfR: str, file_maskL: str, file_maskR: str, file_surfL_inflated=None, file_surfR_inflated=None, maskValue=0, dataType='Surface', templateFormat='HCP', logFile=None)

Prepare a brain surface variable to store surface shape (vertices and faces), and brain masks for useful vertices

Parameters:

• file_surfL – file that stores the surface shape information of the left hemisphere, including vertices and faces

• file_surfR – file that stores the surface shape information of the right hemisphere, including vertices and faces

• file_maskL – file that stores the mask information of the left hemisphere, a 1D 0-1 vector

• file_maskR – file that stores the mask information of the right hemisphere, a 1D 0-1 vector

• file_surfL_inflated – file that stores the inflated surface shape information of the left hemisphere, including vertices and faces

• file_surfR_inflated – file that stores the inflated surface shape information of the right hemisphere, including vertices and faces

• maskValue – 0 or 1, 0 means 0s in mask files are useful vertices, otherwise vice versa. maskValue=0 for medial wall in HCP data, and maskValue=1 for brain masks

• dataType – ‘Surface’

• templateFormat – ‘HCP’, ‘FreeSurfer’, ‘3D Matrix’

• logFile –

Returns:

Brain_Surface: a structure with keys Data_Type, Data_Format, Shape (including L and R), Shape_Inflated (if used), Mask (including L and R)

Yuncong Ma, 12/4/2023

Module.Data_Input.compute_brain_template(dataType: str, templateFormat: str, file_surfL=None, file_surfR=None, file_maskL=None, file_maskR=None, file_mask_vol=None, file_overlayImage=None, maskValue=0, file_surfL_inflated=None, file_surfR_inflated=None, logFile=None)

Prepare a brain surface variable to store surface shape (vertices and faces), and brain masks for useful vertices

Parameters:

• dataType – ‘Surface’, ‘Volume’, ‘Surface-Volume’

• templateFormat – ‘HCP’, ‘FreeSurfer’, ‘3D Matrix’

• file_surfL – file that stores the surface shape information of the left hemisphere, including vertices and faces

• file_surfR – file that stores the surface shape information of the right hemisphere, including vertices and faces

• file_maskL – file that stores the mask information of the left hemisphere, a 1D 0-1 vector

• file_maskR – file that stores the mask information of the right hemisphere, a 1D 0-1 vector

• file_surfL_inflated – file that stores the inflated surface shape information of the left hemisphere, including vertices and faces

• file_surfR_inflated – file that stores the inflated surface shape information of the right hemisphere, including vertices and faces

• file_mask_vol – file of a mask file for volume-based data type

• file_overlayImage – file of a background image for visualizing volume-based results

• maskValue – 0 or 1, 0 means 0s in mask files are useful vertices, otherwise vice versa. maskValue=0 for medial wall in HCP data, and maskValue=1 for brain masks

• logFile –

Returns:

Brain_Template: a structure with keys Data_Type, Data_Format, Shape (including L and R), Shape_Inflated (if used), Mask (including L and R) for surface type a structure with keys Data_Type, Data_Format, Mask, Overlay_Image

Yuncong Ma, 12/4/2023

Module.Data_Input.load_brain_template(file_Brain_Template: str, logFile=None)

Load a brain template file

Parameters:

• file_Brain_Template – directory of the brain_template file, in json format. Python cannot read the MATLAB version

• logFile – directory of a log file

Returns:

Brain_Template: nested dictionary storing information and matrices of brain template. Matrices are converted to np.ndarray

Yuncong Ma, 12/4/2023

Module.Data_Input.load_fmri_scan(file_scan_list: str, dataType: str, dataFormat: str, nTPoints=9999, Reshape=False, Brain_Mask=None, Normalization=None, Concatenation=True, logFile=None)

Updated on 07/28/2024: return imaging data header FY Updated on 08/01/2024: add number of time points to be sampled Load one or multiple fMRI scans, and concatenate them into a single 2D matrix along the time dimension Optional normalization can be added for each scan before concatenation

Parameters:

• file_scan_list – Directory of a single txt file storing fMRI file directories, or a directory of a single scan file

• dataType – ‘Surface’, ‘Volume’, ‘Surface-Volume’

• dataFormat – ‘HCP Surface (.cifti, *.mat)’, ‘MGH Surface (.mgh)’, ‘MGZ Surface (.mgz)’, ‘Volume (.nii, .nii.gz, *.mat)’, ‘HCP Surface-Volume (.cifti)’, ‘HCP Volume (*.cifti)’

• Reshape – False or True, whether to reshape 4D volume-based fMRI data to 2D

• Brain_Mask – None or a brain mask [X Y Z]

• Normalization – False, ‘vp-vmax’

• Concatenation – True, False

• logFile – a log file to save the output

Returns:

Data: a 2D or 4D NumPy array [dim_time dim_space]

By Yuncong Ma, 2/8/2024

Module.Data_Input.load_json_setting(file_setting: str)

Load setting variable or others in a json file, also support gzip

Parameters:

file_setting – Directory of a json setting file

Returns:

Setting

By Yuncong Ma, 10/11/2023

Module.Data_Input.load_matlab_array(file_matlab: str, variable_name: str)

Load a single matlab variable with variable name into np array This support both matrix and cell vector

Parameters:

• file_matlab – string

• variable_name – string

Returns:

data as nparray

By Yuncong Ma, 9/6/2023

Module.Data_Input.load_matlab_single_array(file_matlab: str)

Load a matlab file with only one variable stored This support both matrix and cell vector

Parameters:

file_matlab – string

Returns:

data as nparray

By Yuncong Ma, 9/6/2023

Module.Data_Input.load_matlab_single_variable(file_matlab: str)

Load a matlab file with only one variable stored This support both matrix and cell vector

Parameters:

file_matlab – string

Returns:

data as its original format

By Yuncong Ma, 9/24/2023

Module.Data_Input.load_txt_list(file_txt: str)

Read a txt file containing rows of strings This code can read large txt files

Parameters:

file_txt –

Returns:

list, ndarray of strings

Yuncong Ma, 12/6/2023

Module.Data_Input.ndarray_list(data: ndarray, n_digit=2)

Convert a numpy array to a list with digits truncation

Parameters:

• data – np.ndarray, 1D or 2D matrix

• n_digit – digits reserved

Returns:

a list

Module.Data_Input.normalize_data(data, algorithm='vp', normalization='vmax')

Normalize data by algorithm and normalization settings

Parameters:

• data – data in 2D matrix [dim_time, dim_space]

• algorithm – ‘z’ ‘gp’ ‘vp’

• normalization – ‘n2’ ‘n1’ ‘rn1’ ‘g’ ‘vmax’

Returns:

data

Consistent to MATLAB function normalize_data(X, algorithm, normalization) ‘vp’ is to shift each vector to all non-negative ‘vmax’ is to normalize each vector by its max value

By Yuncong Ma, 12/12/2023

Module.Data_Input.output_FN(FN: ndarray, file_output: str, file_brain_template: str, dataFormat='Volume (*.nii, *.nii.gz, *.mat)', logFile=None, Cheader=None, Nheader=None)

Modified on 07/28/2024 with header information by FY Output FN results in a format matching the input fMRI files

Parameters:

• FN – FN matrix in 2D for surface or surface-volume, 4D for volume, or file directory of a saved FN in .mat, or a tuple of file directories

• file_output – str when FN is ndarray, None when FN is str or tuple of str

• file_brain_template – directory of a brain template file matching pNet requirement

• dataFormat – ‘HCP Surface (.cifti, *.mat)’, ‘MGH Surface (.mgh)’, ‘MGZ Surface (.mgz)’, ‘Volume (.nii, *.nii.gz, *.mat)’,

• logFile – a str

Yuncong Ma, 10/18/2023

Module.Data_Input.print_description_scan_info(logFile: str)

Print the description of scan info

Parameters:

logFile – a string of directory

Yuncong Ma, 9/28/2023

Module.Data_Input.print_log(message: str, logFile=None, style='a', stop=False)

print out message in terminal or logfiles

Parameters:

• message – A string

• logFile – None or a directory of a log file

• style – ‘a’, ‘w’, ‘w+’ or others used for function open

• stop – False or True, if True, the code execution will be stopped

Yuncong Ma, 10/2/2023

Module.Data_Input.reshape_FN(FN: ndarray, dataType: str, Brain_Mask: ndarray, Volume_Order=None, logFile=None)

reshape_fmri_data(scan_data: np.ndarray, dataType: str, Brain_Mask: np.ndarray, logFile=None) If dataType is ‘Volume’ Reshape 4D FNs [X Y Z dim_time] or [X Y Z] into 2D matrix [dim_time dim_space], extracting voxels in Brain_Mask Reshape 2D FNs back to 4D for storage and visualization

Parameters:

• FN – 4D 3D, or 2D matrix [X Y Z K] [dim_space K]

• dataType – ‘Surface’, ‘Volume’, or ‘Surface-Volume’

• Brain_Mask – 3D matrix

• Volume_Order – index order in the volume

• logFile –

Returns:

reshaped_FN: 2D matrix if input is 4D, vice versa

Yuncong Ma, 11/19/2023

Module.Data_Input.reshape_fmri_data(scan_data: np.ndarray, dataType: str, Brain_Mask: np.ndarray, logFile=None)

Reshape 4D volume fMRI data [X Y Z dim_time] into 2D matrix [dim_time dim_space] Reshape 2D fMRI data back to 4D volume type

Parameters:

• scan_data – 4D or 2D matrix [X Y Z dim_time] [dim_time dim_space]

• dataType – ‘Surface’, ‘Volume’, or ‘Surface-Volume’

• Brain_Mask – 3D matrix

• logFile –

Returns:

reshaped_data: 2D matrix if input is 4D, vice versa

Yuncong Ma, 10/5/2023

Module.Data_Input.save_brain_template(dir_pnet_dataInput: str, Brain_Template, logFile=None)

Save the Brain_Template.mat and Brain_Template.json.zip

Parameters:

• dir_pnet_dataInput – the directory of the Data Input folder

• Brain_Template – a structure created by function compute_brain_template

• logFile – ‘Automatic’, None, or a file directory

Yuncong Ma, 1/10/2024

Module.Data_Input.set_data_precision(data_precision: str)

Set the data format and eps Support single, float32, double, float64

Parameters:

data_precision – ‘float32’ or ‘float64’ in python, ‘single’ or ‘double’ in MATLAB

Returns:

np_float, np_eps

By Yuncong Ma, 9/6/2023

Module.Data_Input.set_data_precision_torch(data_precision: str)

Set the data format and eps Support single, float32, double, float64

Parameters:

data_precision – ‘float32’ or ‘float64’ in python, ‘single’ or ‘double’ in MATLAB

Returns:

torch_float, torch_eps

By Yuncong Ma, 9/6/2023

Module.Data_Input.setup_brain_template(dir_pnet_dataInput: str, file_Brain_Template=None, dataType='Surface', templateFormat='HCP', file_surfL=None, file_surfR=None, file_maskL=None, file_maskR=None, file_mask_vol=None, file_overlayImage=None, maskValue=0, file_surfL_inflated=None, file_surfR_inflated=None, logFile='Automatic')

Parameters:

• dir_pnet_dataInput – the directory of the Data Input folder

• file_Brain_Template – file directory or the content of a brain template

• dataType – ‘Surface’, ‘Volume’, ‘Surface-Volume’

• templateFormat – ‘HCP’, ‘FreeSurfer’, ‘3D Matrix’

• file_surfL – file that stores the surface shape information of the left hemisphere, including vertices and faces

• file_surfR – file that stores the surface shape information of the right hemisphere, including vertices and faces

• file_maskL – file that stores the mask information of the left hemisphere, a 1D 0-1 vector

• file_maskR – file that stores the mask information of the right hemisphere, a 1D 0-1 vector

• file_surfL_inflated – file that stores the inflated surface shape information of the left hemisphere, including vertices and faces

• file_surfR_inflated – file that stores the inflated surface shape information of the right hemisphere, including vertices and faces

• file_mask_vol – file of a mask file for volume-based data type

• file_overlayImage – file of a background image for visualizing volume-based results

• maskValue – 0 or 1, 0 means 0s in mask files are useful vertices, otherwise vice versa. maskValue=0 for medial wall in HCP data, and maskValue=1 for brain masks

• logFile – ‘Automatic’, None, or a txt formatted file directory

Yuncong Ma, 2/12/2024

Module.Data_Input.setup_cifti_volume(file_cifti: str, file_output: str, logFile=None)

setup a nifti file for the volume parts in CIFTI

Parameters:

• file_cifti – a CIFTI file ending with .dtseries.nii

• file_output – a NIFTI file ending with .nii or .nii.gz, it stores index order starting from 1

• logFile – a string

Yuncong Ma, 10/6/2023

Module.Data_Input.setup_result_folder(dir_pnet_result: str)

Setup sub-folders in the pNet results folder to store setting and results Including, Data_Input, FN_Computation, Group_FN, Personalized_FN, Quality_Control, Statistics

Parameters:

dir_pnet_result –

Returns:

dir_pnet_dataInput, dir_pnet_FNC, dir_pnet_gFN, dir_pnet_pFN, dir_pnet_QC, dir_pnet_STAT

Yuncong Ma, 9/29/2023

Module.Data_Input.setup_scan_info(dir_pnet_dataInput: str, file_scan: str, file_subject_ID=None, file_subject_folder=None, file_group=None, Combine_Scan=False)

Set up a few txt files for labeling scans file_scan contains the directories of all fMRI scans file_subject_ID contains the subject ID information for each corresponding fMRI scans in file_scan file_subject_folder contains the sub-folder names for each corresponding fMRI scans in file_scan, in order to store results for each fMRI scan or combined scans file_Group_ID contains the group ID for each corresponding fMRI scans in file_scan, in order to do desired sampling based on the group information

Parameters:

• dir_pnet_dataInput – directory of the Data_Input folder or a directory to save Scan_List.txt, Subject_ID.txt, Subject_Folder.txt and Group_ID.txt

• dataType – ‘Surface’, ‘Volume’

• dataFormat – ‘HCP Surface (.cifti, *.mat)’, ‘MGH Surface (.mgh)’, ‘MGZ Surface (.mgz)’, or ‘Volume (.nii, *.nii.gz, *.mat)’

• file_scan – a txt file that stores directories of all fMRI scans

• file_subject_ID – a txt file that store subject ID information corresponding to fMRI scan in file_scan

• file_subject_folder – a txt file that store subject folder names corresponding to fMRI scans in file_scan

• file_group_ID – a txt file that store group information corresponding to fMRI scan in file_scan

• Combine_Scan – False or True, whether to combine multiple scans for the same subject

• logFile – None, ‘Automatic’, or a file directory, for a txt formatted log file

Yuncong Ma, 11/30/2023

Module.Data_Input.write_json_setting(setting, file_setting: str)

Write setting parameter in json format, also support gzip

Parameters:

• setting – a json based variable

• file_setting – Directory of a json setting file

Returns:

none

By Yuncong Ma, 10/9/2023

Module.FN_Computation module

Module.FN_Computation.bootstrap_scan(dir_output: str, file_scan: str, file_subject_ID: str, file_subject_folder: str, file_group=None, combineScan=0, samplingMethod='Subject', sampleSize=10, nBS=50, logFile=None)

prepare bootstrapped scan file lists

Parameters:

• dir_output – directory of a folder to store bootstrapped files

• file_scan – a txt file that stores directories of all fMRI scans

• file_subject_ID – a txt file that store subject ID information corresponding to fMRI scan in file_scan

• file_subject_folder – a txt file that store subject folder names corresponding to fMRI scans in file_scan

• file_group_ID – a txt file that store group information corresponding to fMRI scan in file_scan

• combineScan – 0 or 1, whether to combine multiple fMRI scans for each subject

• samplingMethod – ‘Subject’ or ‘Group_Subject’. Uniform sampling based subject ID, or group and then subject ID

• sampleSize – number of subjects selected for each bootstrapping run

• nBS – number of runs for bootstrap

• logFile – directory of a txt file

Returns:

None

Yuncong Ma, 10/2/2023

Scan based sampling added on Dec 13, 20204

Module.FN_Computation.check_gFN(gFN: ndarray, method='SR-NMF', logFile=None)

Check the values in gFNs to ensure compatibility to the desired FN model

Parameters:

• gFN – group level FNs, 2D matrix for surface type [V K], 4D matrix for volume type [X Y Z K], where K is the number of FNs

• method – ‘SR-NMF’

• logFile – directory of the log file

Yuncong Ma, 9/27/2023

Module.FN_Computation.compute_gNb(Brain_Template, logFile=None)

Prepare a graph neighborhood variable, using indices as its sparse representation

Parameters:

• Brain_Template – a structure variable with keys ‘Data_Type’, ‘Template_Format’, ‘Shape’, ‘Brain_Mask’. If Brain_Template.Data_Type is ‘Surface’, Shape contains L and R, with vertices and faces as sub keys. Brain_Mask contains L and R. If Brain_Template.Data_Type is ‘Volume’, Shape is None, Brain_Mask is a 3D 0-1 matrix, Overlay_Image is a 3D matrix If Brain_Template.Data_Type is ‘Surface-Volume’, It includes fields from both ‘Surface’ and ‘Volume’, ‘Brain_Mask’ is renamed to be ‘Surface_Mask’ and ‘Volume_Mask’

• logFile –

Returns:

gNb: a 2D matrix [N, 2], which labels the non-zero elements in a graph. Index starts from 1

Yuncong Ma, 11/13/2023

Module.FN_Computation.construct_Laplacian_gNb(gNb: np.ndarray, dim_space, vxI=0, X=None, alphaL=10, normW=1, dataPrecision='double')

construct Laplacian matrices for Laplacian spatial regularization term

Parameters:

• gNb – graph neighborhood, a 2D matrix [N, 2] storing rows and columns of non-zero elements

• dim_space – dimension of space (number of voxels or vertices)

• vxI – 0 or 1, flag for using the temporal correlation between nodes (vertex, voxel)

• X – fMRI data, a 2D matrix, [dim_time, dim_space]

• alphaL – internal hyper parameter for Laplacian regularization term

• normW – 1 or 2, normalization method for Laplacian matrix W

• dataPrecision – ‘double’ or ‘single’

Returns:

L, W, D: sparse 2D matrices [dim_space, dim_space]

Yuncong Ma, 9/13/2023

Module.FN_Computation.data_fitting_error(X, U, V, deltaVU, dVordU, dataPrecision='double')

Calculate the datat fitting of X’=UV’ with terms

Parameters:

• X – 2D matrix, [Space, Time]

• U – 2D matrix, [Time, k]

• V – 2D matrix, [Space, k]

• deltaVU – 0

• dVordU – 1

• dataPrecision – ‘double’ or ‘single’

Returns:

Fitting_Error

Consistent to MATLAB function fitting_initialize_u(X, U, V, deltaVU, dVordU, dataPrecision) By Yuncong Ma, 9/6/2023

Module.FN_Computation.gFN_NMF(Data, K, gNb, init='random', maxIter=1000, minIter=200, error=1e-08, normW=1, Alpha=2, Beta=30, alphaS=0, alphaL=0, vxI=0, ard=0, eta=0, nRepeat=5, dataPrecision='double', logFile='Log_pFN_NMF.log')

gFN_NMF(Data, K, gNb, maxIter=1000, minIter=30, error=1e-8, normW=1,

Alpha=2, Beta=30, alphaS=0, alphaL=0, vxI=0, ard=0, eta=0, nRepeat=5, dataPrecision=’double’, logFile=’Log_pFN_NMF.log’)

Compute group-level FNs using NMF method

Parameters:

• Data – 2D matrix [dim_time, dim_space], recommend to normalize each fMRI scan before concatenate them along the time dimension

• K – number of FNs

• gNb – graph neighborhood, a 2D matrix [N, 2] storing rows and columns of non-zero elements

• init – ‘nndsvda’: NNDSVD with zeros filled with the average of X (better when sparsity is not desired) #updated on 08/03/2024 ‘random’: non-negative random matrices, scaled with: sqrt(X.mean() / n_components) ‘nndsvd’: Nonnegative Double Singular Value Decomposition (NNDSVD) initialization (better for sparseness) ‘nndsvdar’ NNDSVD with zeros filled with small random values (generally faster, less accurate alternative to NNDSVDa for when sparsity is not desired)

• maxIter – maximum iteration number for multiplicative update

• minIter – minimum iteration in case fast convergence

• error – difference of cost function for convergence

• normW – 1 or 2, normalization method for W used in Laplacian regularization

• Alpha – hyper parameter for spatial sparsity

• Beta – hyper parameter for Laplacian sparsity

• alphaS – internally determined, the coefficient for spatial sparsity based Alpha, data size, K, and gNb

• alphaL – internally determined, the coefficient for Laplacian sparsity based Beta, data size, K, and gNb

• vxI – flag for using the temporal correlation between nodes (vertex, voxel)

• ard – 0 or 1, flat for combining similar clusters

• eta – a hyper parameter for the ard regularization term

• nRepeat – Any positive integer, the number of repetition to avoid poor initialization

• dataPrecision – ‘single’ or ‘float32’, ‘double’ or ‘float64’

• logFile – str, directory of a txt log file

Returns:

gFN, 2D matrix [dim_space, K]

Yuncong Ma, 11/27/2023

Module.FN_Computation.gFN_fusion_NCut(gFN_BS, K, NCut_MaxTrial=100, dataPrecision='double')

Fuses FN results to generate representative group-level FNs

Parameters:

• gFN_BS – FNs obtained from bootstrapping method, FNs are concatenated along the K dimension

• K – Number of FNs, not the total number of FNs obtained from bootstrapping

• NCut_MaxTrial – Max number trials for NCut method

• dataPrecision – ‘double’ or ‘single’

• logFile – str, directory of a txt log file

Returns:

gFNs, 2D matrix [dim_space, K]

Yuncong Ma, 10/2/2023

Module.FN_Computation.initialize_u(X, U0, V0, error=1e-4, maxIter=1000, minIter=30, meanFitRatio=0.1, initConv=1, dataPrecision='double')

Parameters:

• X – data, 2D matrix [dim_time, dim_space]

• U0 – initial temporal component, 2D matrix [dim_time, k]

• V0 – initial spatial component, 2D matrix [dim_space, k]

• error – data fitting error

• maxIter – maximum iteration

• minIter – minimum iteration

• meanFitRatio – a 0-1 scaler, exponential moving average coefficient

• initConv – 0 or 1, flag for convergence

• dataPrecision – ‘double’ or ‘single’

Returns:

U_final: temporal components of FNs, a 2D matrix [dim_time, K]

Consistent to MATLAB function initialize_u(X, U0, V0, error, maxIter, minIter, meanFitRatio, initConv, dataPrecision) By Yuncong Ma, 9/5/2023

Module.FN_Computation.mat_corr(X, Y=None, dataPrecision='double')

Perform corr as in MATLAB, pair-wise Pearson correlation between columns in X and Y

Parameters:

• X – 1D or 2D matrix

• Y – 1D or 2D matrix, or None

• dataPrecision – ‘double’ or ‘single’

X and Y have the same number of rows :return: Corr

Note: this method will use memory as it concatenates X and Y along column direction. By Yuncong Ma, 9/5/2023

Module.FN_Computation.normalize_data(data, algorithm='vp', normalization='vmax', dataPrecision='double')

Normalize data by algorithm and normalization settings

Parameters:

• data – data in 2D matrix [dim_time, dim_space], recommend to use reference mode to save memory

• algorithm – ‘z’ ‘gp’ ‘vp’

• normalization – ‘n2’ ‘n1’ ‘rn1’ ‘g’ ‘vmax’

• dataPrecision – ‘double’ or ‘single’

Returns:

data

Consistent to MATLAB function normalize_data(X, algorithm, normalization, dataPrecision) By Yuncong Ma, 9/8/2023

Module.FN_Computation.normalize_u_v(U, V, NormV, Norm, dataPrecision='double')

Normalize U and V with terms

Parameters:

• U – 2D matrix, [Time, k]

• V – 2D matrix, [Space, k]

• NormV – 1 or 0

• Norm – 1 or 2

• dataPrecision – ‘double’ or ‘single’

Returns:

U, V

Consistent to MATLAB function normalize_u_v(U, V, NormV, Norm, dataPrecision) By Yuncong Ma, 9/5/2023

Module.FN_Computation.pFN_NMF(Data, gFN, gNb, maxIter=1000, minIter=30, meanFitRatio=0.1, error=0.0001, normW=1, Alpha=2, Beta=30, alphaS=0, alphaL=0, vxI=0, initConv=1, ard=0, eta=0, dataPrecision='double', logFile='Log_pFN_NMF.log')

pFN_NMF(Data, gFN, gNb, maxIter=1000, minIter=30,

meanFitRatio=0.1, error=1e-4, normW=1, Alpha=2, Beta=30, alphaS=2, alphaL=10, initConv=1, ard=0, eta=0, dataPrecision=’double’, logFile=’Log_pFN_NMF.log’)

Compute personalized FNs by spatially-regularized NMF method with group FNs as initialization

Parameters:

• Data – 2D matrix [dim_time, dim_space]. Data will be formatted to Tensor and normalized.

• gFN – group level FNs 2D matrix [dim_space, K], K is the number of functional networks. gFN will be cloned

• gNb – graph neighborhood, a 2D matrix [N, 2] storing rows and columns of non-zero elements

• maxIter – maximum iteration number for multiplicative update

• minIter – minimum iteration in case fast convergence

• meanFitRatio – a 0-1 scaler, exponential moving average coefficient, used for the initialization of U when using group initialized V

• error – difference of cost function for convergence

• normW – 1 or 2, normalization method for W used in Laplacian regularization

• Alpha – hyper parameter for spatial sparsity

• Beta – hyper parameter for Laplacian sparsity

• alphaS – internally determined, the coefficient for spatial sparsity based Alpha, data size, K, and gNb

• alphaL – internally determined, the coefficient for Laplacian sparsity based Beta, data size, K, and gNb

• vxI – 0 or 1, flag for using the temporal correlation between nodes (vertex, voxel)

• initConv – flag for convergence of initialization of U

• ard – 0 or 1, flat for combining similar clusters

• eta – a hyper parameter for the ard regularization term

• dataPrecision – ‘single’ or ‘float32’, ‘double’ or ‘float64’

• logFile – str, directory of a txt log file

Returns:

U and V. U is the temporal components of pFNs, a 2D matrix [dim_time, K], and V is the spatial components of pFNs, a 2D matrix [dim_space, K]

Yuncong Ma, 12/13/2023

Module.FN_Computation.robust_normalize_V(V, factor=0.95, dataPrecision='double')

Module.FN_Computation.run_FN_Computation(dir_pnet_result: str)

run the FN Computation module with settings ready in Data_Input and FN_Computation

Parameters:

dir_pnet_result – directory of pNet result folder

Yuncong Ma, 1/11/2024

Module.FN_Computation.setup_pFN_folder(dir_pnet_result: str)

Setup sub-folders in Personalized_FN to

Parameters:

dir_pnet_result – directory of the pNet result folder

Returns:

list_subject_folder_unique: unique subject folder array for getting sub-folders in Personalized_FN

Yuncong Ma, 9/25/2023

Module.FN_Computation_torch module

Module.FN_Computation_torch.run_FN_Computation_torch(dir_pnet_result: str)

run the FN Computation module with settings ready in Data_Input and FN_Computation

Parameters:

dir_pnet_result – directory of pNet result folder

Yuncong Ma, 2/8/2024

Module.FN_Computation_torch.run_FN_computation_torch_cluster(dir_pnet_result: str)

run the FN Computation module in cluster

Parameters:

dir_pnet_result – directory of pNet result folder

Yuncong Ma, 6/5/2024

class Module.FN_Computation_torch.setup_FN_Computation

Bases: object

setup_GIG_ICA(K=17, Combine_Scan=False, file_gFN=None, maxIter=100, a=0.5, Nemda=1, ftol=0.02, error=1e-05, EGv=0.3745672075, ErChuPai=0.6366197723, Parallel=False, Computation_Mode='CPU', N_Thread=1, dataPrecision='double', outputFormat='MAT')

Setup GIG-ICA parameters for obtaining pFNs

Parameters:

• dir_pnet_result – directory of the pNet result folder

• K – number of FNs

• Combine_Scan – False or True, whether to combine multiple scans for the same subject

• file_gFN – directory of a precomputed gFN in .mat format

# model parameters :param maxIter: maximum iteration number for multiplicative update :param a: weighting to the own pFN :param Nemda: step size for iteration :param ftol: tolerance for the objective function :param error: error tolerance for w to obtain pFN :param EGv: constant :param ErChuPai: constant

# computation resource settings :param Parallel: False or True, whether to enable parallel computation :param Computation_Mode: ‘CPU’ :param N_Thread: positive integers, used for parallel computation :param dataPrecision: ‘double’ or ‘single’ :param outputFormat: ‘MAT’, ‘Both’, ‘MAT’ is to save results in FN.mat and TC.mat for functional networks and time courses respectively. ‘Both’ is for both matlab format and fMRI input file format

Returns:

setting: a structure

Yuncong Ma, 2/12/2024

setup_SR_NMF(K=17, Combine_Scan=False, file_gFN=None, init='random', samplingMethod='Scan', sampleSize='Automatic', nBS=50, nTPoints=99999, maxIter=(2000, 500), minIter=200, meanFitRatio=0.1, error=1e-08, normW=1, Alpha=2, Beta=30, alphaS=0, alphaL=0, vxI=0, ard=0, eta=0, nRepeat=5, Parallel=False, Computation_Mode='CPU', N_Thread=1, dataPrecision='double', outputFormat='Both')

Setup SR-NMF parameters to compute gFNs and pFNs

Parameters:

• dir_pnet_result – directory of the pNet result folder

• K – number of FNs

• Combine_Scan – False or True, whether to combine multiple scans for the same subject

• file_gFN – directory of a precomputed gFN in .mat format

# model parameters :param init: ‘nndsvda’: NNDSVD with zeros filled with the average of X (better when sparsity is not desired) #updated on 08/03/2024

‘random’: non-negative random matrices, scaled with: sqrt(X.mean() / n_components) ‘nndsvd’: Nonnegative Double Singular Value Decomposition (NNDSVD) initialization (better for sparseness) ‘nndsvdar’ NNDSVD with zeros filled with small random values (generally faster, less accurate alternative to NNDSVDa for when spars

Parameters:

• samplingMethod – ‘Subject’, ‘Scan’, or ‘Group_Subject’. Uniform sampling based subject ID, or group and then subject ID

• sampleSize – ‘Automatic’ or integer number, number of subjects selected for each bootstrapping run

• nBS – ‘Automatic’ or integer number, number of runs for bootstrap

• maxIter – maximum iteration number for multiplicative update, which can be one number or two numbers for gFN and pFN separately

• minIter – minimum iteration in case fast convergence, which can be one number or two numbers for gFN and pFN separately

• meanFitRatio – a 0-1 scaler, exponential moving average coefficient, used for the initialization of U when using group initialized V

• error – difference of cost function for convergence

• normW – 1 or 2, normalization method for W used in Laplacian regularization

• Alpha – hyper parameter for spatial sparsity

• Beta – hyper parameter for Laplacian sparsity

• alphaS – internally determined, the coefficient for spatial sparsity based Alpha, data size, K, and gNb

• alphaL – internally determined, the coefficient for Laplacian sparsity based Beta, data size, K, and gNb

• vxI – flag for using the temporal correlation between nodes (vertex, voxel)

• ard – 0 or 1, flat for combining similar clusters

• eta – a hyper parameter for the ard regularization term

• nRepeat – Any positive integer, the number of repetition to avoid poor initialization

# computation resource settings :param Parallel: False or True, whether to enable parallel computation :param Computation_Mode: ‘CPU’ :param N_Thread: positive integers, used for parallel computation :param dataPrecision: ‘double’ or ‘single’ :param outputFormat: ‘MAT’, ‘Both’, ‘MAT’ is to save results in FN.mat and TC.mat for functional networks and time courses respectively. ‘Both’ is for both matlab format and fMRI input file format

Returns:

setting: a structure

Yuncong Ma, 2/5/2024

Module.GIG_ICA module

Module.GIG_ICA.GIG_ICA_neg_entropy(x)

Module.GIG_ICA.pFN_GIG_ICA_cluster(dir_pnet_result: str, jobID=1)

Run the GIG-ICA for pFNs in cluster computation

Parameters:

• dir_pnet_result – directory of pNet result folder

• jobID – jobID starting from 1

Returns:

None

Yuncong Ma, 2/2/2024

Module.GIG_ICA.pFN_GIG_ICA_torch(Data, gFN, maxIter=100, a=0.5, Nemda=1, ftol=0.02, error=1e-05, EGv=0.3745672075, ErChuPai=0.6366197723, dataPrecision='double', logFile='Log_pFN_GIG_ICA.log')

Compute personalized FNs using GIG-ICA

Parameters:

• Data – 2D matrix [dim_time, dim_space], numpy.ndarray or torch.Tensor. Data will be formatted to Tensor and normalized.

• gFN – group level FNs 2D matrix [dim_space, K], K is the number of functional networks, numpy.ndarray or torch.Tensor. gFN will be cloned

• maxIter – maximum iteration number for multiplicative update

• a – weighting to the own pFN

• Nemda – step size for iteration

• ftol – tolerance for the objective function

• error – error tolerance for w to obtain pFN

• EGv – constant

• ErChuPai – constant

• dataPrecision – ‘single’ or ‘float32’, ‘double’ or ‘float64’

• logFile – str, directory of a txt log file

Returns:

TC and FN. TC is the temporal components of pFNs, a 2D matrix [dim_time, K], and FN is the spatial components of pFNs, a 2D matrix [dim_space, K]

Yuncong Ma, 2/2/2024

Module.GIG_ICA.setup_GIG_ICA(dir_pnet_result: str, K=17, Combine_Scan=False, file_gFN=None, maxIter=100, a=0.5, Nemda=1, ftol=0.02, error=1e-05, EGv=0.3745672075, ErChuPai=0.6366197723, Parallel=False, Computation_Mode='CPU', N_Thread=1, dataPrecision='double', outputFormat='MAT')

Setup GIG-ICA parameters for obtaining pFNs

Parameters:

• dir_pnet_result – directory of the pNet result folder

• K – number of FNs

• Combine_Scan – False or True, whether to combine multiple scans for the same subject

• file_gFN – directory of a precomputed gFN in .mat format

# model parameters :param maxIter: maximum iteration number for multiplicative update :param a: weighting to the own pFN :param Nemda: step size for iteration :param ftol: tolerance for the objective function :param error: error tolerance for w to obtain pFN :param EGv: constant :param ErChuPai: constant

# computation resource settings :param Parallel: False or True, whether to enable parallel computation :param Computation_Mode: ‘CPU’ :param N_Thread: positive integers, used for parallel computation :param dataPrecision: ‘double’ or ‘single’ :param outputFormat: ‘MAT’, ‘Both’, ‘MAT’ is to save results in FN.mat and TC.mat for functional networks and time courses respectively. ‘Both’ is for both matlab format and fMRI input file format

Returns:

setting: a structure

Yuncong Ma, 2/12/2024

Module.GIG_ICA.update_model_parameter(dir_pnet_result: False, FN_model_parameter, setting=None)

Update the model parameters in setup_SR_NMF for GIG-ICA

Parameters:

• dir_pnet_result –

• setting – obtained from setup_GIG_ICA

• FN_model_parameter – None or a dict containing model parameters listed in setup_GIG_ICA

Returns:

Yuncong Ma, 2/12/2024

Module.Quality_Control module

Module.Quality_Control.compute_quality_control(scan_data: np.ndarray, gFN: np.ndarray, pFN: np.ndarray, dataPrecision='double', logFile=None)

Compute quality control measurements, including spatial correspondence and functional coherence The spatial correspondence ensures one-to-one match between gFNs and pFNs The functional coherence ensures that pFNs gives better data fitting

Parameters:

• scan_data – 2D matrix, [dim_time, dim_space]

• gFN – 2D matrix, [dim_space, K], K is the number of FNs

• pFN – 2D matrix, [dim_space, K], K is the number of FNs

• dataPrecision – ‘double’ or ‘single’

• logFile – None

Returns:

Spatial_Correspondence, Delta_Spatial_Correspondence, Miss_Match, Functional_Coherence, Functional_Coherence_Control

Spatial correspondence is a 2D symmetric matrix [K, K], which measures the spatial correlation between gFNs and pFNs Delta_Spatial_Correspondence is a vector [K, ], which measures minimum difference of spatial correlation between matched and unmatched gFNs and pFNs Miss_Match is a 2D matrix [N, 2]. Each row notes a pair of miss-matched gFN and pFN. Functional_Coherence is a vector [K, ], which measures the weighted average correlation between node-wise fMRI signal in scan_data and time series of pFNs Functional_Coherence_Control is a vector [K, ], which measures the weighted average correlation between node-wise fMRI signal in scan_data and time series of gFNs

Yuncong Ma, 12/6/2023

Module.Quality_Control.print_description_QC(logFile: str)

Print the description of quality control module

Parameters:

logFile –

Yuncong Ma, 9/28/2023

Module.Quality_Control.run_quality_control(dir_pnet_result: str)

Run the quality control module, which computes spatial correspondence and functional coherence The quality control result folder has consistent sub-folder organization as Personalized_FN Quality control results of each scan or combined scans are stored into sub-folders A single matlab file named Result.mat stores all quantitative values, including Spatial_Correspondence: spatial correspondence between pFNs and gFNs Delta_Spatial_Correspondence: the difference between spatial correspondence of matched pFNs-gFNs and miss-matched pFNs-gFNS Miss_Match: A 2D matrix, [N, 2], each row specifies which pFN is miss matched to a different gFN Functional_Coherence: weighted average of Pearson correlation between time series of pFNs and all nodes Functional_Coherence_Control: weighted average of Pearson correlation between time series of gFNs and all nodes A final report in txt format saved in the root directory of quality control folder It summaries the number of miss matched FNs for each failed scan

Parameters:

dir_pnet_result – the directory of pNet result folder

Returns:

None

Yuncong Ma, 12/19/2023

Module.Quality_Control.visualize_quality_control(dir_pnet_result: str)

Visualize the results of quality control for all scans in the dataset :param dir_pnet_result: directory of the pNet result folder :return: None

Yuncong Ma, 2/14/2024

Module.Quality_Control_torch module

Module.Quality_Control_torch.compute_quality_control_torch(scan_data, gFN, pFN, dataPrecision='double', logFile=None)

Compute quality control measurements, including spatial correspondence and functional coherence The spatial correspondence ensures one-to-one match between gFNs and pFNs The functional coherence ensures that pFNs gives better data fitting

Parameters:

• scan_data – 2D matrix, [dim_time, dim_space], np.ndarray or torch.Tensor

• gFN – 2D matrix, [dim_space, K], K is the number of FNs, np.ndarray or torch.Tensor

• pFN – 2D matrix, [dim_space, K], K is the number of FNs, np.ndarray or torch.Tensor

• dataPrecision – ‘double’ or ‘single’

• logFile – None

Returns:

Spatial_Correspondence, Delta_Spatial_Correspondence, Miss_Match, Functional_Coherence, Functional_Coherence_Control

Outputs are numpy.ndarray Spatial correspondence is a 2D symmetric matrix [K, K], which measures the spatial correlation between gFNs and pFNs Delta_Spatial_Correspondence is a vector [K, ], which measures minimum difference of spatial correlation between matched and unmatched gFNs and pFNs Miss_Match is a 2D matrix [N, 2]. Each row notes a pair of miss-matched gFN and pFN. Functional_Coherence is a vector [K, ], which measures the weighted average correlation between node-wise fMRI signal in scan_data and time series of pFNs Functional_Coherence_Control is a vector [K, ], which measures the weighted average correlation between node-wise fMRI signal in scan_data and time series of gFNs

Yuncong Ma, 2/9/2024

Module.Quality_Control_torch.compute_quality_control_torch_cluster(dir_pnet_result: str, jobID=1)

Run the QC in cluster computation

Parameters:

• dir_pnet_result – directory of pNet result folder

• jobID – jobID starting from 1

Returns:

None

Yuncong Ma, 12/1/2023

Module.Quality_Control_torch.run_quality_control_torch(dir_pnet_result: str)

Run the quality control module, which computes spatial correspondence and functional coherence The quality control result folder has consistent sub-folder organization as Personalized_FN Quality control results of each scan or combined scans are stored into sub-folders A single matlab file named Result.mat stores all quantitative values, including Spatial_Correspondence: spatial correspondence between pFNs and gFNs Delta_Spatial_Correspondence: the difference between spatial correspondence of matched pFNs-gFNs and miss-matched pFNs-gFNS Miss_Match: A 2D matrix, [N, 2], each row specifies which pFN is miss matched to a different gFN Functional_Coherence: weighted average of Pearson correlation between time series of pFNs and all nodes Functional_Coherence_Control: weighted average of Pearson correlation between time series of gFNs and all nodes A final report in txt format saved in the root directory of quality control folder It summaries the number of miss matched FNs for each failed scan

Parameters:

dir_pnet_result – the directory of pNet result folder

Returns:

None

Yuncong Ma, 12/20/2023

Module.Quality_Control_torch.run_quality_control_torch_cluster(dir_pnet_result: str)

Run the quality control module in cluster computation

Parameters:

dir_pnet_result – the directory of pNet result folder

Returns:

None

Yuncong Ma, 6/5/2024

Module.SR_NMF module

Module.SR_NMF.compute_gNb(Brain_Template, logFile=None)

Prepare a graph neighborhood variable, using indices as its sparse representation

Parameters:

• Brain_Template – a structure variable with keys ‘Data_Type’, ‘Template_Format’, ‘Shape’, ‘Brain_Mask’. If Brain_Template.Data_Type is ‘Surface’, Shape contains L and R, with vertices and faces as sub keys. Brain_Mask contains L and R. If Brain_Template.Data_Type is ‘Volume’, Shape is None, Brain_Mask is a 3D 0-1 matrix, Overlay_Image is a 3D matrix If Brain_Template.Data_Type is ‘Surface-Volume’, It includes fields from both ‘Surface’ and ‘Volume’, ‘Brain_Mask’ is renamed to be ‘Surface_Mask’ and ‘Volume_Mask’

• logFile –

Returns:

gNb: a 2D matrix [N, 2], which labels the non-zero elements in a graph. Index starts from 1

Yuncong Ma, 11/13/2023

Module.SR_NMF.construct_Laplacian_gNb(gNb: np.ndarray, dim_space, vxI=0, X=None, alphaL=10, normW=1, dataPrecision='double')

construct Laplacian matrices for Laplacian spatial regularization term

Parameters:

• gNb – graph neighborhood, a 2D matrix [N, 2] storing rows and columns of non-zero elements

• dim_space – dimension of space (number of voxels or vertices)

• vxI – 0 or 1, flag for using the temporal correlation between nodes (vertex, voxel)

• X – fMRI data, a 2D matrix, [dim_time, dim_space]

• alphaL – internal hyper parameter for Laplacian regularization term

• normW – 1 or 2, normalization method for Laplacian matrix W

• dataPrecision – ‘double’ or ‘single’

Returns:

L, W, D: sparse 2D matrices [dim_space, dim_space]

Yuncong Ma, 9/13/2023

Module.SR_NMF.construct_Laplacian_gNb_torch(gNb, dim_space, vxI=0, X=None, alphaL=10, normW=1, dataPrecision='double')

Construct Laplacian matrices for Laplacian spatial regularization term

Parameters:

• gNb – graph neighborhood, a 2D matrix [N, 2] storing rows and columns of non-zero elements

• dim_space – dimension of space (number of voxels or vertices)

• vxI – 0 or 1, flag for using the temporal correlation between nodes (vertex, voxel)

• X – fMRI data, a 2D matrix, [dim_time, dim_space]

• alphaL – internal hyper parameter for Laplacian regularization term

• normW – 1 or 2, normalization method for Laplacian matrix W

• dataPrecision – ‘double’ or ‘single’

Returns:

L, W, D: sparse 2D matrices [dim_space, dim_space]

Yuncong Ma, 9/7/2023

Module.SR_NMF.data_fitting_error_torch(X, U, V, deltaVU=0, dVordU=1, dataPrecision='double')

Calculate the datat fitting of X’=UV’ with terms

Parameters:

• X – 2D matrix, [Space, Time]

• U – 2D matrix, [Time, k]

• V – 2D matrix, [Space, k]

• deltaVU – 0

• dVordU – 1

• dataPrecision – ‘double’ or ‘single’

Returns:

Fitting_Error

Consistent to MATLAB function fitting_initialize_u(X, U, V, deltaVU, dVordU, dataPrecision) By Yuncong Ma, 9/6/2023

Module.SR_NMF.fuse_FN_cluster(dir_pnet_result: str)

Run the NCut to fuse FNs in cluster computation

Parameters:

dir_pnet_result – directory of pNet result folder

Returns:

None

Yuncong Ma, 2/2/2024

Module.SR_NMF.gFN_SR_NMF_boostrapping_cluster(dir_pnet_result: str, jobID=1)

Run the NMF for bootstraping in cluster computation

Parameters:

• dir_pnet_result – directory of pNet result folder

• jobID – jobID starting from 1

Returns:

None

Yuncong Ma, 2/2/2024

Module.SR_NMF.gFN_SR_NMF_torch(Data, K, gNb, init='random', maxIter=1000, minIter=200, error=1e-08, normW=1, Alpha=2, Beta=30, alphaS=0, alphaL=0, vxI=0, ard=0, eta=0, nRepeat=5, dataPrecision='double', logFile='Log_gFN_NMF.log')

Compute group-level FNs using NMF method

Parameters:

• Data – 2D matrix [dim_time, dim_space], numpy.ndarray or torch.Tensor, recommend to normalize each fMRI scan before concatenate them along the time dimension

• K – number of FNs

• gNb – graph neighborhood, a 2D matrix [N, 2] storing rows and columns of non-zero elements

• init – ‘nndsvda’: NNDSVD with zeros filled with the average of X (better when sparsity is not desired) #updated on 08/03/2024 ‘random’: non-negative random matrices, scaled with: sqrt(X.mean() / n_components) ‘nndsvd’: Nonnegative Double Singular Value Decomposition (NNDSVD) initialization (better for sparseness) ‘nndsvdar’ NNDSVD with zeros filled with small random values (generally faster, less accurate alternative to NNDSVDa for when sparsity is not desired)

• maxIter – maximum iteration number for multiplicative update

• minIter – minimum iteration in case fast convergence

• error – difference of cost function for convergence

• normW – 1 or 2, normalization method for W used in Laplacian regularization

• Alpha – hyper parameter for spatial sparsity

• Beta – hyper parameter for Laplacian sparsity

• alphaS – internally determined, the coefficient for spatial sparsity based Alpha, data size, K, and gNb

• alphaL – internally determined, the coefficient for Laplacian sparsity based Beta, data size, K, and gNb

• vxI – flag for using the temporal correlation between nodes (vertex, voxel)

• ard – 0 or 1, flat for combining similar clusters

• eta – a hyper parameter for the ard regularization term

• nRepeat – Any positive integer, the number of repetition to avoid poor initialization

• dataPrecision – ‘single’ or ‘float32’, ‘double’ or ‘float64’

• logFile – str, directory of a txt log file

Returns:

gFN, 2D matrix [dim_space, K]

Yuncong Ma, 2/2/2024

Module.SR_NMF.gFN_fusion_NCut_torch(gFN_BS, K, NCut_MaxTrial=100, dataPrecision='double', logFile='Log_gFN_fusion_NCut')

Fuses FN results to generate representative group-level FNs

Parameters:

• gFN_BS – FNs obtained from bootstrapping method, FNs are concatenated along the K dimension

• K – Number of FNs, not the total number of FNs obtained from bootstrapping

• NCut_MaxTrial – Max number trials for NCut method

• dataPrecision – ‘double’ or ‘single’

• logFile – str, directory of a txt log file

Returns:

gFNs, 2D matrix [dim_space, K]

Yuncong Ma, 10/2/2023

Module.SR_NMF.initialize_u_torch(X, U0, V0, error=0.0001, maxIter=1000, minIter=30, meanFitRatio=0.1, initConv=1, dataPrecision='double')

Initialize U with fixed V, used for pFN_NMF

Parameters:

• X – data, 2D matrix [dim_time, dim_space], numpy.ndarray or torch.Tensor

• U0 – initial temporal component, 2D matrix [dim_time, k], numpy.ndarray or torch.Tensor

• V0 – initial spatial component, 2D matrix [dim_space, k], numpy.ndarray or torch.Tensor

• error – data fitting error

• maxIter – maximum iteration

• minIter – minimum iteration

• meanFitRatio – a 0-1 scaler, exponential moving average coefficient

• initConv – 0 or 1, flag for convergence

• dataPrecision – ‘double’ or ‘single’

Returns:

U_final: temporal components of FNs, a 2D matrix [dim_time, K]

Consistent to MATLAB function initialize_u(X, U0, V0, error, maxIter, minIter, meanFitRatio, initConv, dataPrecision) By Yuncong Ma, 9/5/2023

Module.SR_NMF.normalize_data_torch(data, algorithm='vp', normalization='vmax', dataPrecision='double')

Normalize data by algorithm and normalization settings

Parameters:

• data – data in 2D matrix [dim_time, dim_space], numpy.ndarray or torch.Tensor, recommend to use reference mode to save memory

• algorithm – ‘z’ ‘gp’ ‘vp’

• normalization – ‘n2’ ‘n1’ ‘rn1’ ‘g’ ‘vmax’

• dataPrecision – ‘double’ or ‘single’

Returns:

data

Consistent to MATLAB function normalize_data(X, algorithm, normalization, dataPrecision) By Yuncong Ma, 12/12/2023

Module.SR_NMF.normalize_u_v_torch(U, V, NormV, Norm, dataPrecision='double')

Normalize U and V with terms

Parameters:

• U – 2D matrix, [Time, k]

• V – 2D matrix, [Space, k]

• NormV – 1 or 0

• Norm – 1 or 2

• dataPrecision – ‘double’ or ‘single’

Returns:

U, V

Consistent to MATLAB function normalize_u_v(U, V, NormV, Norm, dataPrecision) By Yuncong Ma, 9/5/2023

Module.SR_NMF.pFN_SR_NMF_cluster(dir_pnet_result: str, jobID=1)

Run the SR-NMF for pFNs in cluster computation

Parameters:

• dir_pnet_result – directory of pNet result folder

• jobID – jobID starting from 1

Returns:

None

Yuncong Ma, 2/2/2024

Module.SR_NMF.pFN_SR_NMF_torch(Data, gFN, gNb, maxIter=1000, minIter=30, meanFitRatio=0.1, error=0.0001, normW=1, Alpha=2, Beta=30, alphaS=0, alphaL=0, vxI=0, initConv=1, ard=0, eta=0, dataPrecision='double', logFile='Log_pFN_NMF.log')

Compute personalized FNs by spatially-regularized NMF method with group FNs as initialization

Parameters:

• Data – 2D matrix [dim_time, dim_space], numpy.ndarray or torch.Tensor. Data will be formatted to Tensor and normalized.

• gFN – group level FNs 2D matrix [dim_space, K], K is the number of functional networks, numpy.ndarray or torch.Tensor. gFN will be cloned

• gNb – graph neighborhood, a 2D matrix [N, 2] storing rows and columns of non-zero elements

• maxIter – maximum iteration number for multiplicative update

• minIter – minimum iteration in case fast convergence

• meanFitRatio – a 0-1 scaler, exponential moving average coefficient, used for the initialization of U when using group initialized V

• error – difference of cost function for convergence

• normW – 1 or 2, normalization method for W used in Laplacian regularization

• Alpha – hyper parameter for spatial sparsity

• Beta – hyper parameter for Laplacian sparsity

• alphaS – internally determined, the coefficient for spatial sparsity based Alpha, data size, K, and gNb

• alphaL – internally determined, the coefficient for Laplacian sparsity based Beta, data size, K, and gNb

• vxI – 0 or 1, flag for using the temporal correlation between nodes (vertex, voxel)

• initConv – flag for convergence of initialization of U

• ard – 0 or 1, flat for combining similar clusters

• eta – a hyper parameter for the ard regularization term

• dataPrecision – ‘single’ or ‘float32’, ‘double’ or ‘float64’

• logFile – str, directory of a txt log file

Returns:

U and V. U is the temporal components of pFNs, a 2D matrix [dim_time, K], and V is the spatial components of pFNs, a 2D matrix [dim_space, K]

Yuncong Ma, 12/13/2023

Module.SR_NMF.robust_normalize_V(V, factor=0.95, dataPrecision='double')

Module.SR_NMF.setup_SR_NMF(dir_pnet_result: str, K=17, Combine_Scan=False, file_gFN=None, init='random', samplingMethod='Scan', sampleSize='Automatic', nBS=50, nTPoints=99999, maxIter=(2000, 500), minIter=200, meanFitRatio=0.1, error=1e-08, normW=1, Alpha=2, Beta=30, alphaS=0, alphaL=0, vxI=0, ard=0, eta=0, nRepeat=5, Parallel=False, Computation_Mode='CPU', N_Thread=1, dataPrecision='double', outputFormat='Both')

Setup SR-NMF parameters to compute gFNs and pFNs

Parameters:

• dir_pnet_result – directory of the pNet result folder

• K – number of FNs

• Combine_Scan – False or True, whether to combine multiple scans for the same subject

• file_gFN – directory of a precomputed gFN in .mat format

# model parameters :param init: ‘nndsvda’: NNDSVD with zeros filled with the average of X (better when sparsity is not desired) #updated on 08/03/2024

‘random’: non-negative random matrices, scaled with: sqrt(X.mean() / n_components) ‘nndsvd’: Nonnegative Double Singular Value Decomposition (NNDSVD) initialization (better for sparseness) ‘nndsvdar’ NNDSVD with zeros filled with small random values (generally faster, less accurate alternative to NNDSVDa for when spars

Parameters:

• samplingMethod – ‘Subject’, ‘Scan’, or ‘Group_Subject’. Uniform sampling based subject ID, or group and then subject ID

• sampleSize – ‘Automatic’ or integer number, number of subjects selected for each bootstrapping run

• nBS – ‘Automatic’ or integer number, number of runs for bootstrap

• maxIter – maximum iteration number for multiplicative update, which can be one number or two numbers for gFN and pFN separately

• minIter – minimum iteration in case fast convergence, which can be one number or two numbers for gFN and pFN separately

• meanFitRatio – a 0-1 scaler, exponential moving average coefficient, used for the initialization of U when using group initialized V

• error – difference of cost function for convergence

• normW – 1 or 2, normalization method for W used in Laplacian regularization

• Alpha – hyper parameter for spatial sparsity

• Beta – hyper parameter for Laplacian sparsity

• alphaS – internally determined, the coefficient for spatial sparsity based Alpha, data size, K, and gNb

• alphaL – internally determined, the coefficient for Laplacian sparsity based Beta, data size, K, and gNb

• vxI – flag for using the temporal correlation between nodes (vertex, voxel)

• ard – 0 or 1, flat for combining similar clusters

• eta – a hyper parameter for the ard regularization term

• nRepeat – Any positive integer, the number of repetition to avoid poor initialization

# computation resource settings :param Parallel: False or True, whether to enable parallel computation :param Computation_Mode: ‘CPU’ :param N_Thread: positive integers, used for parallel computation :param dataPrecision: ‘double’ or ‘single’ :param outputFormat: ‘MAT’, ‘Both’, ‘MAT’ is to save results in FN.mat and TC.mat for functional networks and time courses respectively. ‘Both’ is for both matlab format and fMRI input file format

Returns:

setting: a structure

Yuncong Ma, 2/5/2024

Module.SR_NMF.update_model_parameter(dir_pnet_result: str, FN_model_parameter, setting=None)

Update the model parameters in setup_SR_NMF for SR-NMF

Parameters:

• dir_pnet_result –

• setting – obtained from setup_SR_NMF

• FN_model_parameter – None or a dict containing model parameters listed in setup_SR_NMF

Returns:

Yuncong Ma, 6/5/2024

Module.Visualization module

Module.Visualization.assemble_image(file_list_image: tuple, file_output_assembled=<class 'str'>, organization=(0, 10), interval=(50, 5), background=(0, 0, 0))

Parameters:

• file_list_image – a tuple of image directories or image matrices, images must have the same size

• file_output_assembled – output file directory, can be None to get image matrix as output

• organization – number of rows and columns, default is (0, 10) means to set 10 columns with automatic row number

• interval – (50, 5) in default, meaning the interval is 50 by 5 pixels

• background – (0, 0, 0) in default, meaning the background color is black

Returns:

image_assembled (M, N, 3) matrix, if file_output_assembled is None

Yuncong Ma, 11/3/2023

Module.Visualization.brainmap_colorfunction(brain_map: ndarray, colorbar_range_style='Positive_Only', colorbar_range_round=1, threshold=0)

Prepare a colorfunction based a brain_map and other settings

Parameters:

• brain_map – 1D numpy array

• colorbar_range_style – ‘Positive_Only’, ‘All_Range’

• colorbar_range_round – 1, 0, 0.1, 0.01, 10, etc.

• threshold – 0 for automatic setting, or a number

Returns:

color_function, color_range, threshold_value

Yuncong Ma, 2/8/2024

Module.Visualization.color_theme(theme: str, parameter: tuple, darker=False)

Get predefined color functions based color theme and additional parameters. This function matches to Yuncong Ma’s MATLAB function fColor_Theme

Parameters:

• theme – ‘Seed_Map’, ‘Seed_Map_3_Positive’

• parameter – parameters used for that specific theme. It is a tuple or scalar containing of 1-4 real numbers

• darker – False, or a real number in (0, 1)

Returns:

color_function, a 2D ndarray matrix [N, 4], with first column as values, and the rest three as RGB colors

Examples: color_function = color_theme(‘Seed_Map’,(min_CC,Threshold,max_CC)) color_function = color_theme(‘Seed_Map_2’,(min_CC,max_CC)) color_function = color_theme(‘Seed_Map_3’,(min_CC,max_CC)) color_function = color_theme(‘Seed_Map_Positive’,(min_CC,Threshold,max_CC)) color_function = color_theme(‘Seed_Map_Positive_Only’,(min_CC,Threshold,max_CC)) color_function = color_theme(‘Seed_Map_3_Positive’,(min_CC,max_CC)) color_function = color_theme(‘FC’,max_CC) color_function = color_theme(‘Atlas’,Max_Index) color_function = color_theme(‘Gray_Scale’,(Min_Value,Min_Gray,Max_Value,Max_Gray)) color_function = color_theme(‘Rainbow’,(Min_Value,Max_Value))

Yuncong Ma, 12/9/2023

Module.Visualization.colorize(value_map: ndarray, color_function: ndarray)

colorize a 2D value map into a 2D image [X, Y, 3]

Parameters:

• value_map –

• color_function –

Returns:

image_rgb: np.ndarray, np.float32, 0-1

Yuncong Ma, 11/3/2023

Module.Visualization.compress_image(file_compressed_image: str, image_rgb, image_size=(2000, 10000))

Compress an PIL.Image object to a certain size

Parameters:

• file_compressed_image – directory of the output image file

• image_rgb – an Image file or a PIL.Image object

• image_size – (width, height)

Returns:

Yuncong Ma, 12/18/2023

Module.Visualization.large_3view_center(weight_map: ndarray)

Get the view center for a 3D map, maximizing the content in each view

Parameters:

weight_map – a 3D matrix, either 0-1 or weighted

Returns:

Center ndarray: [3, 1]

Yuncong Ma, 2/1/2024

Module.Visualization.merge_mask_LR(mask_LR: dict)

Merge masks of two hemispheres for surface type

Parameters:

mask_LR – a dict containing ‘L’ and ‘R’, each contains a 1D vector, np.ndarray

Returns:

a 1D vector, np.ndarray

Yuncong Ma, 11/6/2023

Module.Visualization.merge_mesh_LR(mesh_LR: dict, offset=array([90, 0, 0]))

Merge two meshes into one

Parameters:

• mesh_LR – a dict containing ‘L’ and ‘R’, in which vertices and faces are stored

• offset – np.ndarray, 3D vector for the distance between the centers of left and right hemispheres

Returns:

mesh, a dict of vertices and faces

Yuncong Ma, 11/6/2023

Module.Visualization.plot_FN_brain_surface_5view(brain_map: ~numpy.ndarray, brain_template, file_output=<class 'str'>, threshold=99, color_function=None, background=(0, 0, 0), figure_organization=(0.6, 1.2, 1, 0.6), view_angle=(1.35, 1.4), hemisphere_offset=90, figure_title=None, title_font_dic={'fontsize': 20, 'fontweight': 'bold'}, colorbar_range_style='Positive_Only', colorbar_scale=100, colorbar_range_round=1, colorbar_label='Loading (%)', figure_size=(10, 50), dpi=50)

Module.Visualization.plot_FN_brain_surface_volume_7view(brain_map: ~numpy.ndarray, brain_template, file_output=<class 'str'>, threshold=99, color_function=None, background=(0, 0, 0), figure_organization=(0.6, 1.2, 1.5, 0.6), view_angle=1.4, hemisphere_offset=90, view_center='max_value', interpolation='nearest', figure_title=None, title_font_dic={'fontsize': 20, 'fontweight': 'bold'}, colorbar_range_style='Positive_Only', colorbar_scale=100, colorbar_range_round=1, colorbar_label='Loading (%)', figure_size=(10, 50), dpi=50, file_setting=None)

Module.Visualization.plot_FN_brain_volume_3view(brain_map: ~numpy.ndarray, brain_template, file_output=<class 'str'>, threshold=99.8, color_function=None, view_center='max_value', figure_organization=(0.4, 3, 0.6), background=(0, 0, 0), figure_title=None, title_font_dic={'fontsize': 20, 'fontweight': 'bold'}, interpolation='nearest', colorbar_range_style='Positive_Only', colorbar_scale=100, colorbar_range_round=1, colorbar_label='Loading (%)', figure_size=(10, 40), dpi=250, file_setting=None)

Module.Visualization.plot_brain_surface(brain_map: ~numpy.ndarray, mesh: dict, mask: ~numpy.ndarray, color_function: ~numpy.ndarray, file_output=<class 'str'>, orientation='medial', view_angle=1.5, mask_color=(0.2, 0.2, 0.2), brain_color=(0.5, 0.5, 0.5), background=(0, 0, 0), figure_size=(500, 400), dpi=25)

Module.Visualization.plot_voxel_map_3view(Anatomy: ndarray, Voxel_Map: ndarray, center: ndarray, color_function: ndarray, rotation=array([0, 0, 0]), organization=array([0, 0, 0]), background=array([0, 0, 0]), interval=0)

Module.Visualization.prepare_BSPolyData(vertices: ndarray, faces: ndarray)

prepare a BSPolyData class

Parameters:

• vertices – 2D matrix [N, 3], N is the number of vertices

• faces – 2D matrix, [N, 3], N is the number of triangle faces, vertex index starts from 0

Returns:

bspolydata, a data class for using brain space surface plot

Yuncong Ma, 11/1/2023

Module.Visualization.prepare_color_map(map_name=<class 'str'>, color_function=<class 'numpy.ndarray'>, N=256, alpha=1, black_transparent=True)

Prepare a color map using color function

Parameters:

• map_name – a color map name for matplotlib.colors.ListedColormap

• color_function – np.ndarray [N, 4], N is the number of predefined color changing points

• N – digitized colors

• alpha – transparency

• black_transparent – True or False, True is to set black color to transparent

Returns:

cmap: a structure created by matplotlib.colors.ListedColormap

Yuncong Ma, 10/30/2023

Module.Visualization.run_Visualization(dir_pnet_result: str)

Run preconfigured visualizations for gFNs and pFNs

Parameters:

dir_pnet_result – directory of the pnet result folder

Returns:

Yuncong Ma, 11/15/2023

Module.Visualization.run_Visualization_cluster(dir_pnet_result: str)

Run preconfigured visualizations for gFNs and pFNs in cluster computation

Parameters:

dir_pnet_result – directory of the pnet result folder

Returns:

Yuncong Ma, 6/5/2024

Module.Visualization.run_gFN_Visualization(dir_pnet_result: str)

Run preconfigured visualizations for gFNs

Parameters:

dir_pnet_result – directory of the pnet result folder

Returns:

Yuncong Ma, 2/8/2024

Module.Visualization.run_pFN_Visualization(dir_pnet_result: str)

Run preconfigured visualizations for pFNs

Parameters:

dir_pnet_result – directory of the pnet result folder

Returns:

Yuncong Ma, 2/8/2024

Module.Visualization.run_pFN_Visualization_cluster(dir_pnet_result: str, jobID=1)

Run preconfigured visualizations for pFNs in cluster computation

Parameters:

• dir_pnet_result – directory of the pnet result folder

• jobID – jobID starting from 1

Returns:

Yuncong Ma, 2/5/2024

Module.Visualization.setup_Visualization(dir_pnet_result: str, synchronized_view=True, synchronized_colorbar=True)

Setup visualization styles, including synchronized display

Parameters:

• dir_pnet_result – directory of pNet result folder

• synchronized_view – True or False, whether to synchronize view centers for volume data between gFNs and pFNs

• synchronized_colorbar – True or False, whether to synchronize color bar between gFNs and pFNs

Returns:

None

Yuncong Ma, 2/8/2024

Module.Visualization.setup_colorbar_style(FN_Method: str)

Module contents