File I/O: Saving & Loading Native Arrays (.npy, .npz, Text)0%

Capstone: Image Manipulation & Monte Carlo Simulation

File I/O: Saving & Loading Native Arrays (.npy, .npz, Text)

Beginner12 min read•Updated: Jul 11, 2026

Efficient File I/O: .npy, .npz, & CSV/Text Files

Scientific workflows require saving intermediate computations, persisting trained model parameters (such as neural network weights), and loading experimental measurement files.

While CSV or text formats are human-readable, they are notoriously slow to parse and consume immense disk space. NumPy provides optimized binary storage formats (.npy and .npz) that read and write data at raw disk bus bandwidth.

1. Comparing Storage Formats

Feature	Plain Text / CSV	Single Binary (`.npy`)	Compressed Archive (`.npz`)
Speed	Slow (Text $\leftrightarrow$ float conversion)	Lightning fast (raw memory dump)	Fast with high compression
File Size	Bloated text characters	Compact binary	Minimal (ZIP compressed)
Metadata	None (dtypes/shapes lost)	Preserves shape & dtype	Preserves multiple shapes/dtypes
Multi-Array	No (single table only)	Single array only	Dictionary of arrays

2. Single Array Persistence: `.npy`

.npy is a simple binary format designed specifically for NumPy ndarrays. It stores the exact shape, dtype, and byte order in a tiny header, followed immediately by the raw binary memory buffer:

1import numpy as np
2import os
3 
4# Create an array of weights
5weights = np.random.randn(1000, 50)
6 
7# Save to disk
8np.save("model_weights.npy", weights)
9print("Saved model_weights.npy (File size:", os.path.getsize("model_weights.npy"), "bytes)")
10 
11# Load back into memory
12loaded_weights = np.load("model_weights.npy")
13print("Loaded shape:", loaded_weights.shape)
14print("Are identical?", np.array_equal(weights, loaded_weights))
15 
16# Clean up temporary file
17os.remove("model_weights.npy")

3. Multiple Arrays and Compression: `.npz`

When saving multiple related arrays—such as training data $X_{train}$, labels $y_{train}$, and test sets $X_{test}, y_{test}$—use np.savez() or np.savez_compressed():

1X_train = np.random.randn(5000, 20)
2y_train = np.random.randint(0, 2, 5000)
3hyperparameters = np.array([0.01, 100.0, 0.9])
4 
5# Save multiple arrays in a compressed ZIP archive
6np.savez_compressed(
7    "dataset_bundle.npz",
8    features=X_train,
9    labels=y_train,
10    config=hyperparameters
11)
12 
13# Load the bundle
14data_bundle = np.load("dataset_bundle.npz")
15print("Stored array keys:", data_bundle.files)  # ['features', 'labels', 'config']
16 
17# Access individual arrays like a dictionary:
18recovered_X = data_bundle['features']
19recovered_y = data_bundle['labels']
20print("Loaded features shape:", recovered_X.shape)
21 
22# Clean up
23os.remove("dataset_bundle.npz")

4. Memory-Mapped Arrays for Huge Datasets (`mmap_mode`)

What if you need to inspect a 50 GB dataset on a laptop with only 16 GB of RAM? Standard loading would trigger an Out-of-Memory (OOM) crash.

NumPy supports Memory Mapping (mmap_mode), which maps the array on disk directly into the virtual memory address space. NumPy reads slices from disk on-demand as they are accessed, without loading the full file into RAM:

1# Create a dummy large file
2large_arr = np.arange(10_000_000, dtype=np.int64)
3np.save("massive_file.npy", large_arr)
4 
5# Open in read-only memory-mapped mode (instantly returns without loading into RAM!)
6mmap_arr = np.load("massive_file.npy", mmap_mode='r')
7print("Mmap shape:", mmap_arr.shape)
8 
9# Read only a small slice from disk on-demand:
10sample_slice = mmap_arr[500_000:500_010]
11print("Read slice on demand:", sample_slice)
12 
13# Clean up
14del mmap_arr  # close file handle
15os.remove("massive_file.npy")

5. Text / CSV Files: `savetxt` and `genfromtxt`

When interoperability with spreadsheets or legacy software requires plain text CSV:

1table = np.array([
2    [1.0, 25.5, 100.0],
3    [2.0, 30.2, 150.0],
4    [3.0, 18.9, 80.0]
5])
6 
7# Save as CSV with header and format
8np.savetxt("records.csv", table, delimiter=",", header="id,temp,pressure", comments="", fmt="%.2f")
9 
10# Load back
11loaded_table = np.genfromtxt("records.csv", delimiter=",", skip_header=1)
12print("Loaded from CSV:
13", loaded_table)
14 
15os.remove("records.csv")

Interactive Quiz• Knowledge Check

Multiple Choice Questions

Question 1

Which file extension represents NumPy's single binary array storage format?

Question 2

How are multiple arrays saved into a single compressed archive file in NumPy?

Question 3

What is the key advantage of memory-mapped arrays using np.load(..., mmap_mode='r')?

Question 4

When loading an .npz file with bundle = np.load('archive.npz'), how do you inspect the names of the saved arrays?

Question 5

Why is .npy substantially faster than CSV for saving and loading numerical data?

Next Lesson

Capstone: Image Manipulation & Monte Carlo Simulation

Continue learning with hands-on practice, examples, and exercises in the upcoming topic.

Related Lessons

Previous Lesson	Next Lesson
Random Number Generation with default_rng & Distributions	Capstone: Image Manipulation & Monte Carlo Simulation

Practice Quiz

Test your understanding of this lesson with 5 questions. Each question has one correct answer.

Previous Next

PreviousPrevNext

MSK Institute • Shikohabad

Loading Page Content...

Preparing courses, live labs, and learning resources.सामग्री लोड हो रही है, कृपया प्रतीक्षा करें...

Connecting serverPlease wait...

💻 Practical Labs🚀 Free Demo📜 Verified Certificates

Capstone: Image Manipulation & Monte Carlo Simulation

File I/O: Saving & Loading Native Arrays (.npy, .npz, Text)

Beginner12 min read•Updated: Jul 11, 2026

Study Materials Cheatsheet

Efficient File I/O: .npy, .npz, & CSV/Text Files

Scientific workflows require saving intermediate computations, persisting trained model parameters (such as neural network weights), and loading experimental measurement files.

1. Comparing Storage Formats

Feature	Plain Text / CSV	Single Binary (`.npy`)	Compressed Archive (`.npz`)
Speed	Slow (Text $\leftrightarrow$ float conversion)	Lightning fast (raw memory dump)	Fast with high compression
File Size	Bloated text characters	Compact binary	Minimal (ZIP compressed)
Metadata	None (dtypes/shapes lost)	Preserves shape & dtype	Preserves multiple shapes/dtypes
Multi-Array	No (single table only)	Single array only	Dictionary of arrays

2. Single Array Persistence: `.npy`

.npy is a simple binary format designed specifically for NumPy ndarrays. It stores the exact shape, dtype, and byte order in a tiny header, followed immediately by the raw binary memory buffer:

1import numpy as np
2import os
3 
4# Create an array of weights
5weights = np.random.randn(1000, 50)
6 
7# Save to disk
8np.save("model_weights.npy", weights)
9print("Saved model_weights.npy (File size:", os.path.getsize("model_weights.npy"), "bytes)")
10 
11# Load back into memory
12loaded_weights = np.load("model_weights.npy")
13print("Loaded shape:", loaded_weights.shape)
14print("Are identical?", np.array_equal(weights, loaded_weights))
15 
16# Clean up temporary file
17os.remove("model_weights.npy")

3. Multiple Arrays and Compression: `.npz`

When saving multiple related arrays—such as training data $X_{train}$, labels $y_{train}$, and test sets $X_{test}, y_{test}$—use np.savez() or np.savez_compressed():

1X_train = np.random.randn(5000, 20)
2y_train = np.random.randint(0, 2, 5000)
3hyperparameters = np.array([0.01, 100.0, 0.9])
4 
5# Save multiple arrays in a compressed ZIP archive
6np.savez_compressed(
7    "dataset_bundle.npz",
8    features=X_train,
9    labels=y_train,
10    config=hyperparameters
11)
12 
13# Load the bundle
14data_bundle = np.load("dataset_bundle.npz")
15print("Stored array keys:", data_bundle.files)  # ['features', 'labels', 'config']
16 
17# Access individual arrays like a dictionary:
18recovered_X = data_bundle['features']
19recovered_y = data_bundle['labels']
20print("Loaded features shape:", recovered_X.shape)
21 
22# Clean up
23os.remove("dataset_bundle.npz")

4. Memory-Mapped Arrays for Huge Datasets (`mmap_mode`)

What if you need to inspect a 50 GB dataset on a laptop with only 16 GB of RAM? Standard loading would trigger an Out-of-Memory (OOM) crash.

1# Create a dummy large file
2large_arr = np.arange(10_000_000, dtype=np.int64)
3np.save("massive_file.npy", large_arr)
4 
5# Open in read-only memory-mapped mode (instantly returns without loading into RAM!)
6mmap_arr = np.load("massive_file.npy", mmap_mode='r')
7print("Mmap shape:", mmap_arr.shape)
8 
9# Read only a small slice from disk on-demand:
10sample_slice = mmap_arr[500_000:500_010]
11print("Read slice on demand:", sample_slice)
12 
13# Clean up
14del mmap_arr  # close file handle
15os.remove("massive_file.npy")

5. Text / CSV Files: `savetxt` and `genfromtxt`

When interoperability with spreadsheets or legacy software requires plain text CSV:

1table = np.array([
2    [1.0, 25.5, 100.0],
3    [2.0, 30.2, 150.0],
4    [3.0, 18.9, 80.0]
5])
6 
7# Save as CSV with header and format
8np.savetxt("records.csv", table, delimiter=",", header="id,temp,pressure", comments="", fmt="%.2f")
9 
10# Load back
11loaded_table = np.genfromtxt("records.csv", delimiter=",", skip_header=1)
12print("Loaded from CSV:
13", loaded_table)
14 
15os.remove("records.csv")

Interactive Quiz• Knowledge Check

Multiple Choice Questions

Question 1

Which file extension represents NumPy's single binary array storage format?

Question 2

How are multiple arrays saved into a single compressed archive file in NumPy?

Question 3

What is the key advantage of memory-mapped arrays using np.load(..., mmap_mode='r')?

Question 4

When loading an .npz file with bundle = np.load('archive.npz'), how do you inspect the names of the saved arrays?

Question 5

Why is .npy substantially faster than CSV for saving and loading numerical data?

Next Lesson

Capstone: Image Manipulation & Monte Carlo Simulation

Continue learning with hands-on practice, examples, and exercises in the upcoming topic.

Related Lessons

Previous Lesson	Next Lesson
Random Number Generation with default_rng & Distributions	Capstone: Image Manipulation & Monte Carlo Simulation

Practice Quiz

Test your understanding of this lesson with 5 questions. Each question has one correct answer.

Previous Next

Efficient File I/O: .npy, .npz, & CSV/Text Files

1. Comparing Storage Formats

2. Single Array Persistence: .npy

3. Multiple Arrays and Compression: .npz

4. Memory-Mapped Arrays for Huge Datasets (mmap_mode)

5. Text / CSV Files: savetxt and genfromtxt

Multiple Choice Questions

Capstone: Image Manipulation & Monte Carlo Simulation

Related Lessons

Practice Quiz

Loading Page Content...

Efficient File I/O: .npy, .npz, & CSV/Text Files

1. Comparing Storage Formats

2. Single Array Persistence: .npy

3. Multiple Arrays and Compression: .npz

4. Memory-Mapped Arrays for Huge Datasets (mmap_mode)

5. Text / CSV Files: savetxt and genfromtxt

Multiple Choice Questions

Capstone: Image Manipulation & Monte Carlo Simulation

Related Lessons

Practice Quiz

2. Single Array Persistence: `.npy`

3. Multiple Arrays and Compression: `.npz`

4. Memory-Mapped Arrays for Huge Datasets (`mmap_mode`)

5. Text / CSV Files: `savetxt` and `genfromtxt`

2. Single Array Persistence: `.npy`

3. Multiple Arrays and Compression: `.npz`

4. Memory-Mapped Arrays for Huge Datasets (`mmap_mode`)

5. Text / CSV Files: `savetxt` and `genfromtxt`