Access NCBI GEO for gene expression/genomics data. Search/download microarray and RNA-seq datasets (GSE, GSM, GPL), retrieve SOFT/Matrix files, for transcriptomics and expression analysis.

GEO Database Overview The Gene Expression Omnibus (GEO) is NCBI's public repository for high throughput gene expression and functional genomics data. GEO contains over 264,000 studies with more than 8 million samples from both array based and sequence based experiments. When to Use This Skill This skill should be used when searching for gene expression datasets, retrieving experimental data, downloading raw and processed files, querying expression profiles, or integrating GEO data into computational analysis workflows. Core Capabilities 1. Understanding GEO Data Organization GEO organizes data hierarchically using different accession types: Series (GSE): A complete experiment with a set of related samples Example: GSE123456 Contains experimental design, samples, and overall study information Largest organizational unit in GEO Current count: 264,928+ series Sample (GSM): A single experimental sample or biological replicate Example: GSM987654 Contains individual sample data, protocols, and metadata Linked to platforms and series Current count: 8,068,632+ samples Platform (GPL): The microarray or sequencing platform used Example: GPL570 (Affymetrix Human Genome U133 Plus 2.0 Array) Describes the technology and probe/feature annotations Shared across multiple experiments Current count: 27,739+ platforms DataSet (GDS): Curated collections with consistent formatting Example: GDS5678 Experimentally comparable samples organized by study design Processed for differential analysis Subset of GEO data (4,348 curated datasets) Ideal for quick comparative analyses Profiles: Gene specific expression data linked to sequence features Queryable by gene name or annotation Cross references to Entrez Gene Enables gene centric searches across all studies 2. Searching GEO Data GEO DataSets Search: Search for studies by keywords, organism, or experimental conditions: GEO Profiles Search: Find gene specific expression patterns: Advanced Search Patterns: 3. Retrieving GEO Data with GEOparse (Recommended) GEOparse is the primary Python library for accessing GEO data: Installation: Basic Usage: Working with Expression Data: Accessing Supplementary Files: Filtering and Subsetting Data: 4. Using NCBI E utilities for GEO Access E utilities provide lower level programmatic access to GEO metadata: Basic E utilities Workflow: Batch Processing with E utilities: 5. Direct FTP Access for Data Files FTP URLs for GEO Data: GEO data can be downloaded directly via FTP: Using wget or curl for Downloads: 6. Analyzing GEO Data Quality Control and Preprocessing: Differential Expression Analysis: Correlation and Clustering Analysis: 7. Batch Processing Multiple Datasets Download and Process Multiple Series: Meta Analysis Across Studies: Installation and Setup Python Libraries Configuration Set up NCBI E utilities access: Common Use Cases Transcriptomics Research Download gene expression data for specific conditions Compare expression profiles across studies Identify differentially expressed genes Perform meta analyses across multiple datasets Drug Response Studies Analyze gene expression changes after drug treatment Identify biomarkers for drug response Compare drug effects across cell lines or patients Build predictive models for drug sensitivity Disease Biology Study gene expression in disease vs. normal tissues Identify disease associated expression signatures Compare patient subgroups and disease stages Correlate expression with clinical outcomes Biomarker Discovery Screen for diagnostic or prognostic markers Validate biomarkers across independent cohorts Compare marker performance across platforms Integrate expression with clinical data Key Concepts SOFT (Simple Omnibus Format in Text): GEO's primary text based format containing metadata and data tables. Easily parsed by GEOparse. MINiML (MIAME Notation in Markup Language): XML format for GEO data, used for programmatic access and data exchange. Series Matrix: Tab delimited expression matrix with samples as columns and genes/probes as rows. Fastest format for getting expression data. MIAME Compliance: Minimum Information About a Microarray Experiment standardized annotation that GEO enforces for all submissions. Expression Value Types: Different types of expression measurements (raw signal, normalized, log transformed). Always check platform and processing methods. Platform Annotation: Maps probe/feature IDs to genes. Essential for biological interpretation of expression data. GEO2R Web Tool For quick analysis without coding, use GEO2R: Web based statistical analysis tool integrated into GEO Accessible at: https://www.ncbi.nlm.nih.gov/geo/geo2r/?acc=GSExxxxx Performs differential expression analysis Generates R scripts for reproducibility Useful for exploratory analysis before downloading data Rate Limiting and Best Practices NCBI E utilities Rate Limits: Without API key: 3 requests per second With API key: 10 requests per second Implement delays between requests: time.sleep(0.34) (no API key) or time.sleep(0.1) (with API key) FTP Access: No rate limits for FTP downloads Preferred method for bulk downloads Can download entire directories with wget r GEOparse Caching: GEOparse automatically caches downloaded files in destdir Subsequent calls use cached data Clean cache periodically to save disk space Optimal Practices: Use GEOparse for series level access (easiest) Use E utilities for metadata searching and batch queries Use FTP for direct file downloads and bulk operations Cache data locally to avoid repeated downloads Always set Entrez.email when using Biopython Resources references/geo reference.md Comprehensive reference documentation covering: Detailed E utilities API specifications and endpoints Complete SOFT and MINiML file format documentation Advanced GEOparse usage patterns and examples FTP directory structure and file naming conventions Data processing pipelines and normalization methods Troubleshooting common issues and error handling Platform specific considerations and quirks Consult this reference for in depth technical details, complex query patterns, or when working with uncommon data formats. Important Notes Data Quality Considerations GEO accepts user submitted data with varying quality standards Always check platform annotation and processing methods Verify sample metadata and experimental design Be cautious with batch effects across studies Consider reprocessing raw data for consistency File Size Warnings Series matrix files can be large ( 1 GB for large studies) Supplementary files (e.g., CEL files) can be very large Plan for adequate disk space before downloading Consider downloading samples incrementally Data Usage and Citation GEO data is freely available for research use Always cite original studies when using GEO data Cite GEO database: Barrett et al. (2013) Nucleic Acids Research Check individual dataset usage restrictions (if any) Follow NCBI guidelines for programmatic access Common Pitfalls Different platforms use different probe IDs (requires annotation mapping) Expression values may be raw, normalized, or log transformed (check metadata) Sample metadata can be inconsistently formatted across studies Not all series have series matrix files (older submissions) Platform annotations may be outdated (genes renamed, IDs deprecated) Additional Resources GEO Website: https://www.ncbi.nlm.nih.gov/geo/ GEO Submission Guidelines: https://www.ncbi.nlm.nih.gov/geo/info/submission.html GEOparse Documentation: https://geoparse.readthedocs.io/ E utilities Documentation: https://www.ncbi.nlm.nih.gov/books/NBK25501/ GEO FTP Site: ftp://ftp.ncbi.nlm.nih.gov/geo/ GEO2R Tool: https://www.ncbi.nlm.nih.gov/geo/geo2r/ NCBI API Keys: https://ncbiinsights.ncbi.nlm.nih.gov/2017/11/02/new api keys for the e utilities/ Biopython Tutorial: https://biopython.org/DIST/docs/tutorial/Tutorial.html

geo-database