Modules_File_I

Python's with statement provides a clean, reliable way to work with files by guaranteeing that resources are released properly, even when errors occur. Understanding file modes, reading methods, and path handling gives you a solid foundation for any file-based workflow.

• Opening Files and File Modes
• The first argument to open() is the file path as a string or Path object. This path can be absolute or relative to the current working directory, giving you flexibility in how you locate files.
• The mode argument controls whether the file is opened for reading, writing, or appending. Common modes include 'r' for reading, 'w' for writing (which overwrites existing content), and 'a' for appending new content to the end.
• An optional encoding argument (e.g., encoding='utf-8') prevents character-encoding errors on different platforms. Without specifying encoding, Python uses the system default, which can vary between Windows, macOS, and Linux and cause subtle bugs.
• The with Statement Context Manager
• Use with open(path, mode) as f: to open a file within a managed context. The with statement calls the file object's __enter__ method on entry and __exit__ method on exit, handling setup and teardown automatically.
• The file object is bound to the alias after as and is available inside the block. You use this alias (commonly f) to call read, write, and other file methods throughout the indented body of the with block.
• The file is closed automatically when the with block exits, regardless of errors. This eliminates the need for explicit f.close() calls and prevents resource leaks that can occur if an exception is raised mid-operation.
• Reading Text Files
• f.read() returns the entire file content as a single string. This is convenient for small files, but loading very large files entirely into memory can be inefficient and should be avoided for large datasets.
• f.readline() reads and returns one line at a time, including the newline character. Calling it repeatedly advances through the file line by line, making it useful when you need precise control over line-by-line processing.
• f.readlines() returns a list of all lines in the file. Each element in the list corresponds to one line and retains its trailing newline, so you often need to call .strip() when processing individual lines.
• Writing Text Files
• f.write(string) writes the given string to the file and returns the number of characters written. Unlike print(), write() does not add a newline automatically, so you must include '\n' explicitly when needed.
• f.writelines(list) writes each item in a list to the file without adding separators. You are responsible for including any newline characters within the list items themselves if line breaks are required in the output.
• Open in append mode ('a') to add content to an existing file without erasing it. If the file does not yet exist, Python creates it automatically, making append mode safe to use even on first run.
• Working with File Paths Using pathlib
• Use Path from pathlib to create platform-independent path objects. Unlike raw strings, Path objects handle forward slashes, backslashes, and other OS-specific differences transparently.
• The / operator joins path components intuitively, e.g., Path('data') / 'results.txt'. This is far more readable than string concatenation and avoids common mistakes such as missing or doubled directory separators.
• Path.exists() returns True if the path points to an existing file or directory. Checking existence before opening prevents FileNotFoundError exceptions and allows your code to handle missing files gracefully.

# Reading a file with the with statement
from pathlib import Path

data_path = Path('data') / 'measurements.txt'

with open(data_path, 'r', encoding='utf-8') as f:
    contents = f.read()
print(contents)

# Writing results to a new file
output_path = Path('data') / 'results.txt'

lines = ['Trial 1: 23.4\n', 'Trial 2: 25.1\n', 'Trial 3: 22.8\n']
with open(output_path, 'w', encoding='utf-8') as f:
    f.writelines(lines)

CSV (Comma-Separated Values) files are one of the most common formats for exchanging tabular data in engineering and science. Python's built-in csv module provides straightforward tools for reading and writing these files without relying on external libraries.

• What Is a CSV File?
• The first row of a CSV file often contains column headers. These headers label each column and are used by tools like csv.DictReader to provide named access to fields rather than relying on positional indexing.
• Fields containing the delimiter or newlines are enclosed in quotation marks. This quoting convention allows complex data — such as addresses or notes — to be stored in a single field without breaking the row structure.
• The delimiter can be customized; tabs, semicolons, and pipes are common alternatives. You specify an alternative delimiter with the delimiter parameter when creating a reader or writer object, making the csv module flexible for many file variants.
• Reading CSV Files with csv.reader
• Pass the open file object to csv.reader to create a reader object. The reader handles the parsing of each line, correctly splitting fields and removing quotation marks around quoted values.
• Iterate over the reader to process each row as a list. Each iteration yields one row as a list of strings, so you will typically need to convert numeric fields explicitly using int() or float().
• Use next(reader) to skip a header row before processing data rows. Calling next() once advances the reader past the first line, preventing header text from being treated as data values.
• Reading CSV Files with csv.DictReader
• DictReader automatically uses the first row as field names unless fieldnames is specified. This removes the need to manually skip the header and makes your code more robust to changes in column order.
• Access row values by column name: row['Temperature']. Named access is more readable and less error-prone than index-based access, especially in files with many columns.
• Each row is returned as a dictionary-like object with consistent key ordering. Because keys come from the header row, your code documents itself — the column names appear directly in the source rather than as opaque index numbers.
• Writing CSV Files with csv.writer
• Create a writer object by passing the open file object to csv.writer. On Windows, open the file with newline='' to prevent the writer from adding extra blank lines between rows.
• Use writer.writerow(list) to write a single row. Each element of the list becomes a field in the output row, and the writer automatically adds quotation marks around fields that contain the delimiter.
• Use writer.writerows(list_of_lists) to write multiple rows at once. This is more efficient than calling writerow() in a loop and is preferred when all the data is available as a collection.
• Writing CSV Files with csv.DictWriter
• Pass fieldnames as a list when creating the DictWriter object. The fieldnames list controls both the order of columns in the output and which dictionary keys are written to the file.
• Call writeheader() before writing data rows to include column labels. Skipping this step produces a CSV without a header, which can confuse downstream tools and make the file harder to interpret.
• Use writerow(dict) or writerows(list_of_dicts) to write data. If a dictionary contains keys not listed in fieldnames, they are ignored by default, giving you control over which columns appear in the output.

import csv
from pathlib import Path

sensor_path = Path('data') / 'sensor_log.csv'

# Reading with DictReader
with open(sensor_path, 'r', encoding='utf-8', newline='') as f:
    reader = csv.DictReader(f)
    readings = []
    for row in reader:
        readings.append({
            'time': row['Time'],
            'temp': float(row['Temperature'])
        })

# Writing results with DictWriter
output_path = Path('data') / 'summary.csv'
fields = ['time', 'temp']

with open(output_path, 'w', encoding='utf-8', newline='') as f:
    writer = csv.DictWriter(f, fieldnames=fields)
    writer.writeheader()
    writer.writerows(readings)

JSON (JavaScript Object Notation) is a lightweight, human-readable format widely used for configuration files, web APIs, and data exchange between applications. Python's built-in json module makes it straightforward to convert between JSON text and native Python data structures.

• What Is JSON?
• JSON supports six data types: strings, numbers, booleans, null, arrays, and objects. This limited but practical type set covers the vast majority of data-exchange needs and maps cleanly onto Python's core types.
• JSON objects correspond closely to Python dictionaries; arrays correspond to lists. This natural mapping means that most JSON data can be read directly into Python and manipulated with familiar dictionary and list operations.
• JSON files are plain text and can be opened and inspected in any text editor. This transparency makes debugging easier compared to binary formats and allows version control systems to show meaningful diffs when configuration files change.
• Parsing JSON with json.loads and json.load
• json.loads(string) parses a JSON-formatted string and returns a Python object. The s in loads stands for "string," distinguishing it from the file-based variant; use it when JSON arrives as text from a network response or variable.
• json.load(file_object) reads and parses JSON directly from an open file. Pairing it with the with statement ensures the file is closed after parsing, keeping resource management clean and consistent.
• JSON objects become Python dicts, arrays become lists, and JSON null becomes None. Understanding this type mapping prevents confusion when you inspect parsed data and find None where the original JSON contained null.
• Serializing Data with json.dumps and json.dump
• json.dumps(object) returns a JSON-formatted string from a Python object. The optional indent parameter (e.g., indent=2) produces pretty-printed output that is easier for humans to read and review.
• json.dump(object, file_object) writes JSON directly to an open file. Like json.load, it is best used inside a with block so the file is properly closed after writing.
• Python dicts, lists, strings, numbers, booleans, and None are natively serializable. As long as your data consists only of these types, no additional configuration is needed to convert it to valid JSON.
• JSON for Configuration Files
• Define default parameters in a .json file and load them at script startup. Storing parameters externally separates configuration from logic, making scripts easier to reuse across different experiments or deployment environments.
• Updating the config file changes script behavior without touching Python code. This is especially valuable in collaborative projects where team members need to adjust parameters without understanding or modifying the underlying implementation.
• Nested JSON objects allow hierarchical grouping of related parameters. For example, sensor settings, output preferences, and algorithm thresholds can each live in their own nested object, making the configuration file self-documenting and organized.
• Limitations of Python's json Module
• Sets, tuples (stored as arrays), and custom class instances are not natively JSON-serializable. Attempting to serialize these types raises a TypeError, so you must convert them to supported types before calling json.dump or json.dumps.
• A custom JSONEncoder subclass can extend serialization to additional types. By overriding the default() method, you can teach the encoder how to handle sets, NumPy arrays, datetime objects, or any other custom type.
• Always validate JSON output when working with mixed Python and NumPy numeric types. NumPy scalars such as np.float64 are not standard Python floats and will raise a TypeError unless converted with float() or handled by a custom encoder.

import json
from pathlib import Path

# Loading a JSON configuration file
config_path = Path('config') / 'experiment.json'

with open(config_path, 'r', encoding='utf-8') as f:
    config = json.load(f)

sample_rate = config['acquisition']['sample_rate_hz']
output_dir  = config['output']['directory']

# Saving results as JSON
results = {
    'trial': 1,
    'mean_temp': 24.7,
    'passed': True,
    'notes': None
}

results_path = Path('data') / 'results.json'

with open(results_path, 'w', encoding='utf-8') as f:
    json.dump(results, f, indent=2)