Lesson 1: Function Design

The problem this lesson solves

Open the recipe-extractor project from the previous modules and you will find something that happens to every codebase left alone for a few weeks. The cleaning logic exists in four places. One version strips whitespace and lowercases. One strips whitespace, lowercases, and removes HTML tags. One does all of that plus collapses repeated newlines. The fourth was written last Tuesday and nobody remembers why it differs.

When a bug turns up in cleaning, fixing it means finding all four, deciding which behaviour was correct, and hoping nothing depended on the wrong one.

This is not a discipline problem. It is what happens when functions are written as somewhere to put code rather than as designed units. This lesson is about designing them, and the four topics that follow build on the result.

One job, honest name, predictable return

Three properties make a function reusable. Miss any one and the function becomes something people copy rather than call.

One job. A function should do a single thing that can be described without the word "and". Consider this:

def process_document(path: Path) -> dict:
    content = path.read_text(encoding="utf-8")
    content = content.strip().lower()
    content = re.sub(r"<[^>]+>", "", content)
    word_count = len(content.split())
    if word_count < 10:
        logger.warning(f"Document {path} is very short")
    save_to_database(path, content, word_count)
    return {"path": str(path), "words": word_count}

Describe it honestly and you get: it reads a file and cleans the text and counts words and logs a warning and writes to a database and returns a summary. Six jobs. You cannot reuse the cleaning without also touching the database. You cannot test the word count without a filesystem. You cannot change the cleaning rules without reading database code.

Split by responsibility:

def read_document(path: Path) -> str:
    return path.read_text(encoding="utf-8")


def clean_text(raw: str) -> str:
    without_tags = re.sub(r"<[^>]+>", "", raw)
    return without_tags.strip().lower()


def count_words(text: str) -> int:
    return len(text.split())

Each of these is now reusable, testable in isolation, and changeable without reading anything else.

Honest name. The name should say what the function does, including the parts you would rather not advertise.

def get_user(user_id: str) -> User:          # dishonest if it also writes a cache entry
def validate_email(address: str) -> str:      # dishonest if it also normalises the address
def load_config() -> Settings:                # dishonest if it also exits the program on failure

A name starting with get, load, or fetch promises a read. If the function also changes something, the name is lying, and the person who calls it in a loop will be surprised. If you cannot name a function honestly in a few words, that is usually a sign it is doing more than one job.

Predictable return. A function should return one kind of thing, always.

def parse_servings(raw: str):
    if not raw:
        return None
    if raw.isdigit():
        return int(raw)
    if "-" in raw:
        return raw.split("-")          # returns a list, sometimes
    return raw                          # returns a string, sometimes

Four possible return types. Every caller must handle all four, and the type checker cannot help because the signature promises nothing. Callers will handle the two cases they happened to encounter in testing.

def parse_servings(raw: str) -> int | None:
    """Return a serving count, or None if the input does not contain one."""
    stripped = raw.strip()
    if not stripped:
        return None
    if stripped.isdigit():
        return int(stripped)
    match = re.match(r"(\d+)\s*-\s*\d+", stripped)   # "4-6" becomes 4
    if match:
        return int(match.group(1))
    return None

Two possible outcomes, both in the signature. The caller handles absence once and is done. Note that "predictable" does not mean "never returns None". It means the set of possible returns is small, declared, and consistent.

Input, Validate, Transform, Output

Most functions that do real work have the same internal shape, and naming that shape makes functions easier to write and much easier to read.

def summarize_document(doc: Document, max_words: int = 100) -> Summary:
    # Input: everything needed arrives as parameters
    
    # Validate: reject what cannot be handled, early
    if max_words <= 0:
        raise ValueError(f"max_words must be positive, got {max_words}")
    if doc.status != "valid":
        raise ValueError(f"Cannot summarize document with status {doc.status!r}")

    # Transform: the actual work, with no surprises left
    words = doc.content.split()
    truncated = " ".join(words[:max_words])
    
    # Output: one constructed result, returned once
    return Summary(
        document_id=doc.id,
        text=truncated,
        was_truncated=len(words) > max_words,
    )

The validation section is the guard clause pattern from the previous module, applied at the top of every function that takes input it does not control. By the time the transform section starts, every assumption has been checked, so the working code has no defensive branches in it.

Two things worth noticing. The transform section is the only part that varies between functions, which means most of a function is predictable structure. And the output is constructed in one place and returned once, rather than being assembled through mutation across several branches, which is much easier to follow.

Parameters, defaults, keyword-only arguments, *args and **kwargs

Positional and keyword arguments. Parameters can be passed by position or by name. Position is concise and keyword is explicit.

def chunk_text(text: str, size: int, overlap: int) -> list[str]: ...

chunk_text(content, 500, 50)                        # what are 500 and 50?
chunk_text(content, size=500, overlap=50)           # clear

The first call is ambiguous at the reading site, and swapping the two numbers produces working code with wrong behaviour. Use keyword arguments for anything that is not obvious from the function name, especially numbers and booleans.

Defaults. A default makes a parameter optional and declares the sensible choice.

def chunk_text(text: str, size: int = 500, overlap: int = 50) -> list[str]: ...

Two rules. Parameters with defaults must come after those without. And the default must never be a mutable object, for the reason covered in the previous modules.

def add_tag(doc: Document, tags: list[str] = []) -> Document:      # shared between calls
def add_tag(doc: Document, tags: list[str] | None = None) -> Document:   # correct

Keyword-only arguments. Placing a bare * in the signature forces everything after it to be passed by name.

def chunk_text(text: str, *, size: int = 500, overlap: int = 50) -> list[str]: ...

chunk_text(content, 500, 50)              # TypeError
chunk_text(content, size=500, overlap=50) # required form

This is worth doing whenever a function takes more than one option, and especially when it takes booleans or several numbers of the same type. It prevents the swapped-argument bug entirely, and it means you can add or reorder options later without breaking existing callers, since nobody is depending on position.

A useful convention: make the data positional and everything else keyword-only. chunk_text(content, size=..., overlap=...) reads well and cannot be called wrongly.

*args and kwargs.** These collect any number of extra positional or keyword arguments.

def log_event(name: str, *args: object, **kwargs: object) -> None:
    print(name, args, kwargs)

log_event("parsed", 1, 2, doc_id="abc", duration_ms=12)
# parsed (1, 2) {'doc_id': 'abc', 'duration_ms': 12}

args is a tuple of the extra positional values. kwargs is a dict of the extra keyword values.

Use them for genuine pass-through, where a function must accept arguments it does not itself understand and hand them onward. This is exactly what decorators do, which is why they appear again in Lesson 3.

def retry(func, *args, **kwargs):
    """Call func with whatever arguments it was given, retrying on failure."""
    for attempt in range(3):
        try:
            return func(*args, **kwargs)
        except TransientError:
            continue
    raise

Do not use them to avoid deciding on a signature. A function declared as def process(**kwargs) documents nothing, autocompletes nothing, and turns a misspelled parameter name into silent default behaviour rather than an error.

Pure versus impure, and why pure parts are testable

A pure function has two properties. Given the same inputs it always returns the same output, and it changes nothing outside itself.

# Pure
def clean_text(raw: str) -> str:
    return re.sub(r"<[^>]+>", "", raw).strip().lower()


# Impure: reads the filesystem
def load_document(path: Path) -> str:
    return path.read_text(encoding="utf-8")


# Impure: writes to a database
def save_document(doc: Document) -> None:
    database.insert(doc)


# Impure: output depends on the clock
def make_document_id() -> str:
    return f"doc_{datetime.now().timestamp()}"


# Impure: calls a network service
def summarize_with_model(text: str) -> str:
    return client.generate(prompt=f"Summarize: {text}")

Anything a function does beyond computing its return value is a side effect. Reading a file, writing to a database, sending a request, printing, mutating a global, and appending to a list that was passed in are all side effects.

Why this distinction earns its place. Pure functions are trivially testable, because a test is a call and an assertion.

def test_clean_text_removes_tags():
    assert clean_text("<p>Hello</p>") == "hello"

No setup, no database, no mocking, no network, no cleanup, and no flakiness. It runs in microseconds and it will produce the same result on every machine forever.

Testing an impure function means arranging the world first. A temporary directory, a test database, a fake HTTP server, a frozen clock. All of that is possible and a later module covers how, but it is slower to write, slower to run, and more fragile.

The design consequence. Side effects cannot be eliminated, since a program that touches nothing does nothing useful. What you can do is push them to the edges and keep the middle pure.

# Impure shell: does the input and output
def process_corpus(folder: Path, output: Path) -> None:
    raw_documents = [p.read_text(encoding="utf-8") for p in folder.iterdir()]

    documents = build_documents(raw_documents)          # pure
    valid, rejected = partition_by_status(documents)    # pure
    report = build_report(valid, rejected)              # pure

    output.write_text(report.to_json())


# Pure core: all the decisions
def build_documents(raw: list[str]) -> list[Document]: ...
def partition_by_status(docs: list[Document]) -> tuple[list[Document], list[Document]]: ...
def build_report(valid: list[Document], rejected: list[Document]) -> Report: ...

Every rule, every edge case, and every judgement lives in the three pure functions, which are tested exhaustively with plain values. The impure function contains no logic worth testing, only the reading and writing, so one integration test covering it is enough.

This shape has a name in wider use, the functional core with an imperative shell, and it is the single most useful structural idea in this lesson.

Composition and pipelines

When functions each do one job and return a predictable type, they connect.

def prepare(raw: str) -> Document:
    cleaned = clean_text(raw)
    normalized = normalize_whitespace(cleaned)
    return build_document(normalized)

That is composition, meaning the output of one function becoming the input of the next. It works only because the types line up: clean_text returns a string, normalize_whitespace takes a string, and so on. Types are what make composition possible, which is why the previous module put so much weight on them.

For a repeated sequence, express the pipeline as data rather than as a chain of calls:

from collections.abc import Callable

TextTransform = Callable[[str], str]

CLEANING_PIPELINE: list[TextTransform] = [
    strip_html,
    normalize_whitespace,
    normalize_unicode,
    lowercase,
]


def apply_pipeline(text: str, transforms: list[TextTransform]) -> str:
    for transform in transforms:
        text = transform(text)
    return text


cleaned = apply_pipeline(raw, CLEANING_PIPELINE)

Callable[[str], str] is a type hint meaning "a function taking one string and returning a string". Naming it TextTransform makes the list's contents self-explanatory.

Three things this buys you. The pipeline is now visible in one place, so a reader can see the whole cleaning process without following four function calls. It is configurable, since a different corpus can use a different list. And it is testable at two levels, meaning each transform on its own and the composed pipeline as a whole.

This also solves the problem the lesson opened with. There are no longer four cleaning implementations. There is one list, and any variation is a different list built from the same tested pieces.

Concept check. You have a function def enrich(doc: Document) -> Document that fetches metadata from an API, adds it to the document, writes an audit log line, and returns the result. Is it pure? What would you change and why?

Answer

It is impure twice over. It calls a network service, so the same input can produce different outputs, and it writes a log line, which is a side effect.

The useful change is to split the fetch from the merge. fetch_metadata(doc_id) -> Metadata is impure and small, containing no logic. merge_metadata(doc, metadata) -> Document is pure and contains all the decisions about how fields combine, what wins on conflict, and what happens when metadata is missing. Those decisions are the part with bugs in it, and they can now be tested with plain values and no network at all.

The audit log is a side effect belonging to the caller rather than to a data transformation. Move it out to the impure shell.