GEM well table

cfg_GEM_wells.tsv in the selected configuration directory has one row per GEM well, that is, one cellranger-arc count output. You fill it in at step 1 of Configure your data, and aggregations select wells by their GEM_well_ID.

Columns

Copy a row of the public example, give it a unique GEM_well_ID, and fill in these columns:

Column What to enter
GEM_well_ID Unique ID used in aggregation_GEM_well_IDs, target names and output folders. Keep it stable and avoid dots.
GEM_well_dataset Groups related GEM wells in the pre-aggregation QC summaries. A processing and reporting group, not a donor identifier.
GEM_well_donor_id Links a single-donor GEM well to its row in the donor metadata table. Use NA for a pooled well, whose donors come from GEM_well_donors_VCF_file.
GEM_well_n_donors Number of donors in the GEM well. Use 1 for a single-donor well; for a pooled well, Vireo assigns nuclei to this many donors.
GEM_well_cellranger_arc_count_dir Locates the Cell Ranger ARC count output for this GEM well. Point to the directory containing outs, not to outs itself.
GEM_well_cellbender_h5_file Gives the CellBender-corrected GEX count file, whose counts then replace the Cell Ranger counts. Use NA to keep the Cell Ranger counts.
GEM_well_donors_VCF_file Donor-genotype VCF that enables demultiplexing of a pooled GEM well. Its sample names become the donor IDs. Use NA for a single-donor well, whose nuclei all receive GEM_well_donor_id.
GEM_well_QC_exclude_list Per-nucleus filters applied before aggregation. NA for the first run; later, complete R expressions separated by ;;.
GEM_well_is_active TRUE to process the GEM well; FALSE for unused rows, including unused demo wells. Every GEM well selected by an active aggregation must also be active.

Add library or batch annotations as extra columns prefixed with GEM_well_, such as GEM_well_multiplex_batch. They become cell metadata: select them for plots with aggregation_categorical_vars or aggregation_continuous_vars, and for batch correction with aggregation_harmony_correction_metadata_col_names. Keep donor phenotypes in the donor metadata table.

Pooled GEM wells can also set GEM_well_cellsnp_min_MAF and GEM_well_cellsnp_min_count, the minimum minor-allele frequency and reads per SNP that cellsnp-lite uses for demultiplexing. Left out or NA, they default to 0.1 and 20.

Cell Ranger inputs

Each GEM_well_cellranger_arc_count_dir must contain:

outs/
├── summary.csv
├── filtered_feature_bc_matrix.h5
├── atac_fragments.tsv.gz
├── atac_fragments.tsv.gz.tbi
└── per_barcode_metrics.csv

The pipeline is tested with the outputs of Cell Ranger ARC 2.0.0 and 2.1.0.

A pooled GEM well also needs atac_possorted_bam.bam in outs/. For CellBender counts, run CellBender remove-background first and give its H5 file in GEM_well_cellbender_h5_file.

The pipeline identifies each well’s Cell Ranger ARC reference by matching the FASTA and GTF hashes in the atac_fragments.tsv.gz header to a reference.json in the repository’s resources/reference_metadata/ folder, so keep that header intact. JSON files for the GRCh38 2020-A, GRCh38 2024-A and mm10 2020-A references are included; for another reference, copy its reference.json into a new subdirectory there.

Pooled GEM wells

A GEM well that pools several donors needs a donor-genotype VCF in GEM_well_donors_VCF_file, the number of donors in GEM_well_n_donors, and NA in GEM_well_donor_id; the Vireo documentation describes the VCF. The pipeline genotypes each nucleus with cellsnp-lite and assigns it with Vireo. Every nucleus receives its most likely single donor, named by the VCF’s sample name, and vireo_type records whether Vireo called it a singlet, a doublet or unassigned; remove doublets and unassigned nuclei with a QC filter such as vireo_type != "singlet".

Without a VCF, no demultiplexing runs and every nucleus receives GEM_well_donor_id, so a pooled well needs its VCF to obtain donors.

Rules checked

When the pipeline starts, it checks that:

  • GEM_well_ID values are unique;
  • every GEM well that an active aggregation selects exists and is active; and
  • each QC filter is a valid R expression.

When the targets of a well run, exactly one reference.json must match its fragments, and all wells of an aggregation must share the same reference. Apart from the keys, no column name may appear both here and in the donor metadata table.

Set QC filters after the first run

Leave GEM_well_QC_exclude_list as NA for the first run. After reviewing the QC distributions at checkpoint 1, enter filters and rerun checkpoint 1. Each filter is a complete R expression over the per-nucleus metadata; separate filters with ;;:

cfg_GEM_wells.tsv
TSS.enrichment < 4 ;; nucleosome_signal > 4 ;; nCount_RNA < 250

A nucleus for which any expression is TRUE is excluded, and each expression is reported as a separate exclusion reason in the checkpoint 1 UpSet and retention plots. Nuclei absent from the GEX count matrix are always excluded, as not_found_in_GEX_matrix, whether or not the list names it. Filters can use the metrics that QC_metric_manifest.tsv lists as available from checkpoint 1, and other per-nucleus columns such as vireo_type. The checkpoint 1 comparison plots show each metric’s distribution and draw simple cutoffs such as these; they are examples, not recommendations for your tissue.

Public example

The table below shows the two public demo wells with the required columns in bold and one optional annotation, GEM_well_cell_sorting. Scroll horizontally, and focus or hover over a column’s i button for its meaning. The committed example contains further inactive rows and annotation columns.

cfg_GEM_wells.tsv
GEM_well_IDGEM well ID. Unique ID used in aggregation_GEM_well_IDs, target names and output folders. Keep it stable and avoid dots. GEM_well_datasetDataset. Groups related GEM wells in the pre-aggregation QC summaries. A processing and reporting group, not a donor identifier. GEM_well_donor_idDonor ID. Links a single-donor GEM well to its row in the donor metadata table. Use NA for a pooled well, whose donors come from GEM_well_donors_VCF_file. GEM_well_n_donorsExpected donors. Number of donors in the GEM well. Use 1 for a single-donor well; for a pooled well, Vireo assigns nuclei to this many donors. GEM_well_cellranger_arc_count_dirCell Ranger ARC count directory. Locates the Cell Ranger ARC count output for this GEM well. Point to the directory containing outs, not to outs itself. GEM_well_cellbender_h5_fileCellBender H5 file. Gives the CellBender-corrected GEX count file, whose counts then replace the Cell Ranger counts. Use NA to keep the Cell Ranger counts. GEM_well_donors_VCF_fileDonor VCF file. Donor-genotype VCF that enables demultiplexing of a pooled GEM well. Its sample names become the donor IDs. Use NA for a single-donor well, whose nuclei all receive GEM_well_donor_id. GEM_well_QC_exclude_listPre-aggregation QC filters. Per-nucleus filters applied before aggregation. NA for the first run; later, complete R expressions separated by ;;. GEM_well_is_activeActive. TRUE to process the GEM well; FALSE for unused rows, including unused demo wells. Every GEM well selected by an active aggregation must also be active. GEM_well_cell_sortingCell or nuclei sorting. Describes sorting or enrichment before library preparation. Example optional metadata column for values such as sorted, depleted, or unsorted material. Add other GEM_well_ columns when your analysis needs them.
healthy_PBMC_human immune_human_dataset pbmc1 1 example_data/healthy_PBMC_human NA NA TSS.enrichment < 4 ;; nucleosome_signal > 4 ;; nCount_RNA < 250 ;; RNA_mito_percent > 10 TRUE granulocytes_removed
controller_PBMC_human immune_human_dataset pbmc2 1 example_data/controller_PBMC_human NA NA TSS.enrichment < 4 ;; nucleosome_signal > 4 ;; nCount_RNA < 250 ;; RNA_mito_percent > 10 TRUE unspecified