🐍Languages

Top 50 Python
interview questions

Data model, generators, decorators, asyncio, GIL and packaging. Open a question to read the answer.

01

Is Python a compiled language or an interpreted language?

Both, in practice. CPython, the reference implementation, first compiles source into bytecode and then interprets that bytecode on a stack-based virtual machine. The compiled bytecode is cached as .pyc files in __pycache__, so unchanged modules skip the compile step on the next import.

The phrase “interpreted language” is therefore a statement about the usual implementation, not about the language itself. Other implementations compile differently: PyPy uses a tracing JIT, Cython and Nuitka translate to C, and CPython 3.13 ships an experimental copy-and-patch JIT that is disabled unless the interpreter is built with --enable-experimental-jit.

import dis

def area(r):
    return 3.14159 * r * r

dis.dis(area)        # shows LOAD_FAST, BINARY_OP, RETURN_VALUE...

Two consequences matter in interviews: there is no separate link step and no static type checking at compile time, so type errors surface at run time, and performance depends mainly on how much work is pushed into C-level built-ins and extension modules rather than on the bytecode loop itself. Python 3.11 and later also specialise bytecode at run time (the “adaptive interpreter”), which is why upgrading alone often speeds code up noticeably.

02

What is a lambda function in Python?

A lambda is an anonymous function defined in a single expression: lambda arguments: expression. It creates a normal function object; the only differences from def are that it has no name (its __name__ is <lambda>), it cannot contain statements, and its body must be one expression whose value is returned implicitly.

orders = [("A-1", 250.0), ("A-2", 80.5), ("A-3", 120.0)]
by_amount = sorted(orders, key=lambda o: o[1], reverse=True)
print(by_amount[0])          # ('A-1', 250.0)

square = lambda x: x * x     # legal but discouraged: use def instead

Lambdas belong where a short callable is passed inline: key= arguments to sorted, min and max, callbacks, or small transformations in map. They support default values, *args and **kwargs, and they close over enclosing variables like any function, which means the usual late-binding caveat applies inside loops.

PEP 8 advises against assigning a lambda to a name, because a def gives a real name in tracebacks and allows a docstring. When the logic needs more than one line, a conditional chain, or a comment, a named function is the right tool.

03

Explain lists tuples sets dictionaries in the context of Python.

These are the four built-in collection types, and each one trades off ordering, mutability and lookup cost differently.

TypeOrderedMutableDuplicatesTypical lookup
listYesYesYesO(1) by index, O(n) by value
tupleYesNoYesO(1) by index, O(n) by value
setNoYesNoO(1) average membership
dictInsertion order (3.7+)YesUnique keysO(1) average by key

Lists are dynamic arrays: appending at the end is amortised O(1), but inserting or deleting at the front is O(n), which is why collections.deque exists for queues. Tuples are fixed-size and hashable when their elements are, so they can be dictionary keys or set members and double as lightweight records. Sets and dictionaries are hash tables; their elements and keys must be hashable, and set operations (&, |, -, ^) make them the natural tool for deduplication and membership testing.

seen = set()
unique = [x for x in [3, 1, 3, 2, 1] if not (x in seen or seen.add(x))]
print(unique)                      # [3, 1, 2]
point = (10, 20)                   # hashable, usable as a dict key
distances = {point: 22.4}

Dictionaries are the backbone of the language itself: object attributes, keyword arguments and module namespaces are all dictionaries. Since Python 3.7 they are guaranteed to preserve insertion order, and dict | other merges two of them (3.9+).

04

In a real project using Python, how would you apply, test, or troubleshoot lists tuples sets dictionaries?

Apply. In an order-processing service I pick the collection by access pattern, not by habit. Incoming line items arrive as a list because order matters for the invoice. Lookups of product metadata by SKU go through a dictionary built once at startup, so each enrichment is O(1) instead of scanning a list per item. Deduplicating customer e-mail addresses for a campaign is a set operation, and set(new_ids) -set(existing_ids) gives me the delta in one line. Immutable configuration such as (host, port) pairs becomes tuples so they can serve as dictionary keys and cannot be mutated by accident.

Test. Unit tests assert behaviour that depends on the type’s guarantees: a parametrised pytest case checks that duplicates are collapsed, that ordering is preserved where the contract promises it, and that a lookup for a missing key raises KeyError or returns the documented default. For large datasets I add a benchmark test with pytest-benchmark to catch a regression from O(1) to O(n), which typically appears when someone replaces a set membership check with x in some_list.

Troubleshoot. The classic symptom is a request that gets slower as data grows. cProfile shows time concentrated in list.__contains__ or in list.insert(0, ...); the fix is a set, a dictionary or a deque. The second common bug is a mutable default argument (def f(items=[])) that accumulates state across calls, visible as data from one request leaking into the next; the fix is items=None and creating the list inside the function. “dictionary changed size during iteration” means a loop mutated the dictionary it was iterating; iterate over list(d.items()) instead.

An important consideration is that a tuple is only as immutable as its contents: ([1], 2) cannot be a dictionary key because the inner list is unhashable.

05

Explain list comprehensions in the context of Python.

A list comprehension builds a list from an iterable in one expression: [expression for item in iterable if condition]. It replaces the explicit loop-append pattern, runs faster than the equivalent loop because the append is executed in C rather than as a method call, and states intent in a single line.

prices = {"pen": 1.5, "book": 12.0, "bag": 35.0}
expensive = [name.upper() for name, p in prices.items() if p > 10]
print(expensive)                           # ['BOOK', 'BAG']

matrix = [[1, 2], [3, 4]]
flat = [x for row in matrix for x in row]  # nested: outer loop first
print(flat)                                # [1, 2, 3, 4]

Key points:

  • The loop variable is local to the comprehension and does not leak into the enclosing scope, unlike a plain for loop. Python 3.12 inlines comprehensions into the surrounding function (PEP 709) for speed, but preserves that isolation.
  • The same syntax produces sets ({x for x in ...}), dictionaries ({k: v for ...}) and, with parentheses, a lazy generator expression.
  • Multiple for clauses nest left to right, and if filters can appear after any of them.
  • A conditional expression can appear in the value position: [x if x > 0 else 0 for x in data].

A comprehension becomes the wrong tool once it needs side effects, more than two for clauses, or logic that no longer fits on one or two lines; at that point an explicit loop or a helper function reads better.

06

In a real project using Python, how would you apply, test, or troubleshoot list comprehensions?

Apply. In a reporting pipeline I use comprehensions for the pure transform-and-filter steps: converting a list of database rows into DTOs, selecting the subset of records that failed validation, or building a lookup dictionary with {row.id: row for row in rows}. When the input is a large file or a streaming query result, the same expression written with parentheses becomes a generator expression, so I can feed sum(...), any(...) or csv.writer.writerows(...) without materialising a list. Anything that calls out to a database or an API inside the loop stays an explicit for statement so the side effects and error handling are visible.

Test. Comprehensions are usually wrapped in small pure functions, which makes them straightforward to test with parametrised inputs: empty input, a single element, and an input where every element is filtered out. For data-heavy code I compare the result against a naive loop implementation in a test so a later “optimisation” cannot silently change semantics. ruff rules such as C4 flag unnecessary comprehensions (list(x for x in y)) and PERF rules flag loops that should be comprehensions.

Troubleshoot. The two production problems I have actually seen are memory and readability. A comprehension over a multi-million-row cursor materialised everything at once and the worker was OOM-killed; tracemalloc pointed to the line, and the fix was a generator expression plus chunked processing. The second is a triple-nested comprehension with two conditionals that nobody could debug; rewriting it as a loop with named intermediate variables fixed the logic error in minutes. Also watch for calling an expensive function twice ([f(x) for x in d if f(x)]); the walrus operator ([y for x in d if (y := f(x))]) evaluates it once.

The key consideration is that a comprehension signals “pure transformation” to readers, so hiding side effects inside one misleads the next maintainer.

07

Explain iterators and generators in the context of Python.

An iterator is any object that implements __iter__() returning itself and __next__() returning the next value or raising StopIteration. An iterable is anything that iter() can turn into an iterator: lists, dictionaries, files, and user classes with __iter__. The for loop, unpacking, sum() and comprehensions all drive this protocol.

A generator is the simplest way to write an iterator. Any function containing yield returns a generator object when called; its body runs lazily, pausing at each yield and resuming on the next request, with local state preserved between steps.

def read_batches(path, size):
    batch = []
    with open(path, encoding="utf-8") as f:
        for line in f:
            batch.append(line.rstrip("\n"))
            if len(batch) == size:
                yield batch
                batch = []
    if batch:
        yield batch

for rows in read_batches("events.log", 1000):
    process(rows)                 # memory stays flat regardless of file size

Why they matter:

  • Laziness: values are produced on demand, so pipelines over gigabytes of data run in constant memory and can be infinite (itertools.count()).
  • Composition: generators chain naturally, and yield from delegates to a sub-generator.
  • Iterators are single-pass; once exhausted they stay exhausted, which trips people who iterate a map or zip object twice.
  • send(), throw() and close() turn a generator into a coroutine-like object, which is the mechanism asyncio was originally built on.

Generator expressions (x * 2 for x in data) give the same laziness without a function definition.

08

In a real project using Python, how would you apply, test, or troubleshoot iterators and generators?

Apply. In an ETL job that ingests multi-gigabyte CSV exports, every stage is a generator: one reads lines, one parses them into records, one validates, and one yields insert batches of 500 rows. The stages compose with plain function calls, so adding a filter is a one-line change and peak memory stays at a few megabytes. The same pattern paginates a REST API: a generator hides the next_page_token loop and the caller just iterates. For resource cleanup I wrap generators that hold files or connections in contextlib.closing or make the pipeline a context manager, because a generator abandoned mid-way only releases its resources when garbage-collected.

Test. I test each stage with small in-memory iterables rather than files, asserting on list(stage(input)). Laziness itself is tested by passing an infinite source like itertools.count() and using itertools.islice to take the first few items; if the stage secretly calls list() on its input, the test hangs or is killed by a pytest-timeout. Exhaustion behaviour is covered by asserting that a second iteration yields nothing where that is the contract.

Troubleshoot. The most frequent bug report is “the function returns nothing”: someone added yield to a function and the caller still expects a list, so the generator object is never consumed. The opposite is passing a generator to two consumers; the second sees an empty sequence. Memory profiles that still grow usually reveal a hidden sorted() or list() in one stage that forces materialisation. GeneratorExit tracebacks at interpreter shutdown mean a finally block in a generator did something non-trivial while being closed.

One consideration: a generator with try/finally must not yield inside the finally, and errors raised inside a generator surface at the consumer’s next() call, which can be far from the faulty code.

09

Explain decorators and closures in the context of Python.

A closure is a function that captures variables from the enclosing scope and keeps them alive after that scope has returned. Python stores the captured variables in cell objects reachable through func.__closure__. A decorator is a callable that takes a function (or class) and returns a replacement; the @name syntax is sugar for target = name(target). Most decorators are closures, because the wrapper they return must remember the original function.

import functools, time

def retry(times):                         # decorator factory
    def decorator(func):
        @functools.wraps(func)            # keeps name, docstring, signature
        def wrapper(*args, **kwargs):
            for attempt in range(times):
                try:
                    return func(*args, **kwargs)
                except ConnectionError:
                    time.sleep(0.1 * 2 ** attempt)
            return func(*args, **kwargs)  # last attempt raises normally
        return wrapper
    return decorator

@retry(times=3)
def fetch_rate(currency): ...

Key points:

  • functools.wraps copies metadata and sets __wrapped__, so inspect and debuggers can see through the wrapper.
  • A closure reads the variable, not its value at definition time; rebinding requires nonlocal. This late binding is why [lambda: i for i in range(3)] all return 2.
  • Decorators run at import time, once per definition, while the wrapper runs on every call. Expensive setup belongs in the decorator, not in the wrapper.
  • Class-based decorators implement __call__, and decorators stack bottom-up: the one nearest the function is applied first.

The standard library uses the pattern everywhere: functools.lru_cache, functools.cache, property, contextlib.contextmanager and dataclasses.dataclass.

10

In a real project using Python, how would you apply, test, or troubleshoot decorators and closures?

Apply. In a payments API decorators carry the cross-cutting concerns that must be identical on every endpoint: @require_scope("payments:write") for authorisation, @timed("payments.capture") to emit a latency histogram, and @idempotent(key=lambda req: req.headers["Idempotency-Key"]) to short-circuit retries. Each one is a closure over its configuration, so the business handler stays free of plumbing. Closures also replace tiny single-method classes: a make_rate_limiter(limit) returning a function that remembers its token bucket is clearer than a class with one method.

Test. I test the decorator in isolation by applying it to a trivial function inside the test and asserting on behaviour: the wrapped function is called the expected number of times (a Mock as the target), the metric emitter received the right name, and wrapped.__name__ and __doc__ survive, which guards against someone deleting functools.wraps. Because decorators run at import, a syntax or configuration error in one breaks every module that imports it, so a smoke test that imports every package catches that early.

Troubleshoot. Symptoms I have debugged: FastAPI endpoints losing their request-parameter signature because a decorator dropped functools.wraps, which made the framework see *args, **kwargs; inspect.signature immediately showed the problem. Tracebacks ending in wrapper() with no useful frame are solved by __wrapped__ and by keeping wrappers thin. A cache decorator holding references to large request objects produced a slow memory leak; tracemalloc snapshots pointed at the cache dictionary, and bounding it with lru_cache(maxsize=...) fixed it. Finally, UnboundLocalError inside a closure means an assignment turned a captured variable into a local; nonlocal is the fix.

The important consideration is that decorators hide control flow, so each one should do exactly one thing and be documented where it is defined.

11

Explain args kwargs parameters in the context of Python.

*args and **kwargs are conventions, not keywords: a single star in a parameter list collects surplus positional arguments into a tuple, and a double star collects surplus keyword arguments into a dictionary. The names args and kwargs are only a habit; the stars do the work. At call sites the same syntax does the reverse, unpacking an iterable into positional arguments and a mapping into keyword arguments.

def log_event(event, *tags, level="info", **fields):
    print(event, tags, level, fields)

log_event("login", "auth", "web", user="ana", ip="10.0.0.1")
#=> login ('auth', 'web') info {'user': 'ana', 'ip': '10.0.0.1'}

params = {"level": "warn", "user": "bob"}
log_event("logout", *["auth"], **params)
#=> logout ('auth',) warn {'user': 'bob'}

The full parameter order is: positional-only parameters, then /, regular parameters, *args (or a bare *), keyword-only parameters, and finally **kwargs. A bare * forces everything after it to be passed by keyword, which is how sorted(data, key=..., reverse=True) refuses sorted(data, f, True). The / marker (Python 3.8+) makes parameters positional-only, protecting their names from becoming part of the public API.

Typical uses are forwarding arguments unchanged in decorators and subclasses (super().__init__(*args, **kwargs)), building flexible factory functions, and merging configuration dictionaries. The cost is lost discoverability: an IDE cannot tell what **kwargs accepts, so explicit parameters are preferable in public interfaces, with TypedDict plus Unpack (3.12 typing) available when the dictionary shape must be typed.

12

In a real project using Python, how would you apply, test, or troubleshoot args kwargs parameters?

Apply. The legitimate production uses are narrow. In a service layer I use *args, **kwargs in decorators and in thin adapter classes that forward to a vendor SDK, so that a new SDK parameter needs no change in my wrapper. Builders for database clients accept **options and pass them through to the driver. Everywhere else I prefer explicit, keyword-only parameters: def create_order(*, customer_id: int, lines: list[Line], currency: str = "EUR") cannot be called with arguments in the wrong order, which has saved us from real bugs where two integer IDs were swapped.

Test. For forwarding code the test patches the downstream callable with Mock(autospec=True) and asserts mock.assert_called_once_with(expected_args, key=value), which checks that both the tuple and the dictionary arrive intact. For keyword-only APIs a test asserts that positional misuse raises TypeError. Static typing does a lot of the work: mypy --strict rejects a call that passes an unexpected keyword to a function with explicit parameters, and ParamSpec lets a decorator preserve the wrapped signature so those checks still apply through the wrapper.

Troubleshoot. TypeError: got multiple values for argument 'x' means a value was passed both positionally and via **kwargs, usually from a caller merging dictionaries. TypeError: got an unexpected keyword argument after a library upgrade means a parameter was renamed and the pass-through hid it until run time. Silently ignored options are the worst case: a function accepting **kwargs and never reading them swallows typos like timout=5; I add an explicit check that kwargs is empty after known keys are popped. Mutating the kwargs dictionary before forwarding is fine because it is a fresh dictionary per call, but mutating objects inside it is not.

The key consideration is that **kwargs trades compile-time clarity for flexibility; the trade is worth it only at genuine pass-through boundaries.

13

Explain deep copy versus shallow copy in the context of Python.

Assignment never copies in Python; b = a binds a second name to the same object. A shallow copy creates a new outer container whose elements are the same objects as the original, while a deep copy recursively copies every nested object so the two structures share nothing mutable.

AspectShallow copyDeep copy
Howcopy.copy(x), list(x), x[:], dict(x), x.copy()copy.deepcopy(x)
Nested objectsShared with the originalIndependent clones
CostO(n) over the top levelWalks the whole graph; handles cycles via memo
Safe whenElements are immutable (ints, str, tuples)Elements are mutable and will be changed
import copy

order = {"id": 7, "lines": [{"sku": "A", "qty": 1}]}
shallow = copy.copy(order)
deep = copy.deepcopy(order)

shallow["lines"][0]["qty"] = 99      # also changes order["lines"]
deep["lines"][0]["qty"] = 5          # order unaffected
print(order["lines"][0]["qty"])      # 99

deepcopy keeps a memo dictionary so shared references and cycles are copied once and preserved, and classes can customise either operation with __copy__ and __deepcopy__. Immutable objects such as strings and tuples of immutables are returned as-is by both functions because copying them would be pointless. Python 3.13 adds copy.replace(), which creates a modified copy of objects that define __replace__, including dataclasses and named tuples, covering the common “copy with one field changed” case without a full deep copy.

14

In a real project using Python, how would you apply, test, or troubleshoot deep copy versus shallow copy?

Apply. In a pricing engine the base configuration is loaded once and then specialised per tenant. Each tenant gets copy.deepcopy(base_config) before overrides are applied, because the configuration is a nested dictionary of lists and a shallow copy would let one tenant’s override leak into every other tenant. Shallow copies are deliberately used where they are enough and cheaper: returning list(self._items) from a getter protects the internal list from external append calls without duplicating the items themselves. For dataclass-based domain objects I prefer dataclasses.replace() or copy.replace() (3.13) over deep copying, and I make value objects frozen=True so the question disappears entirely.

Test. Tests for anything that copies state assert isolation explicitly: copy the object, mutate a nested field in the copy, and assert the original is unchanged. A second test asserts the reverse direction. For custom __deepcopy__ implementations I also test that cycles do not recurse forever and that resources which must not be duplicated, such as a database connection held by the object, are shared rather than cloned.

Troubleshoot. The symptom is “data changes by itself”: a request handler sees fields modified by a previous request, or a default fixture in tests is polluted by an earlier test. I confirm it with id() on the nested objects, which match between the supposedly independent copies, then fix with deepcopy or by restructuring so the mutable state is created fresh. The opposite problem is performance: deepcopy on large object graphs per request showed up at the top of a cProfile listing, and the fix was to copy only the branch being modified or to switch to immutable structures. deepcopy failing with “cannot pickle” errors means the graph contains a lock, socket or generator; implementing __deepcopy__ to skip that attribute resolves it.

The key consideration is that deep copying is a sign of shared mutable state; immutability usually removes the need for it.

15

Explain identity versus equality in the context of Python.

Identity asks whether two names refer to the same object in memory; equality asks whether two objects have the same value. is tests identity by comparing id() values and cannot be overridden. == calls __eq__, which classes define to express value semantics; the default implementation inherited from object falls back to identity, so instances of a plain class compare equal only to themselves.

a = [1, 2, 3]
b = [1, 2, 3]
c = a
print(a == b, a is b)     # True False  (same value, different objects)
print(a == c, a is c)     # True True   (same object)

x = 1000; y = 1000
print(x == y, x is y)     # True, and 'is' is implementation-dependent

Rules that follow from this:

  • Compare to singletons with is: if value is None, is True, is NotImplemented. There is exactly one None, so identity is both correct and fastest.
  • Compare values with ==. Code like if count is 0 works only because CPython caches small integers from -5 to 256, and Python 3.8+ emits a SyntaxWarning for is with a literal.
  • Defining __eq__ sets __hash__ to None unless you define it too, because equal objects must hash equally for sets and dictionaries to work.
  • Containers compare elements with == but first check identity as a shortcut, which is why nan in [nan] is True even though nan == nan is False.

String interning and constant folding make "abc" is "abc" appear to work in the REPL; relying on it is a bug waiting for a different input source.

16

In a real project using Python, how would you apply, test, or troubleshoot identity versus equality?

Apply. Domain objects in a billing system get explicit equality semantics. Value objects such as Money(amount, currency) are frozen dataclasses, so __eq__ and __hash__ are generated from their fields and two equal amounts behave identically in sets and as dictionary keys. Entities such as Invoice compare by business identifier only, so I write __eq__ to compare self.id and keep __hash__ consistent with it, which lets a changed invoice still match its cached version. Sentinel objects are compared with is: _MISSING = object() as a default distinguishes “argument omitted” from “argument was None”, and if result is _MISSING is unambiguous.

Test. Each class with custom equality gets tests for reflexivity, symmetry, inequality with a different type (must return NotImplemented, not raise), and the hash contract: hash(a) == hash(b) whenever a == b. A set-based test, inserting two equal objects and asserting the length is one, catches the forgotten __hash__. For ORMs I test that an object loaded twice in different sessions compares equal, since identity will differ.

Troubleshoot. is comparisons that break after a refactor are the classic incident: a status is "active" check passed for years because the literal was interned, then failed once the value came from a database driver as a fresh string. ruff rule F632 and the interpreter’s SyntaxWarning flag this. TypeError: unhashable type after adding __eq__ to a class means __hash__ was implicitly removed. Duplicate entries in a set of objects point at an __eq__ and __hash__ pair that disagree, often because the hash includes a mutable field that changed after insertion. For NumPy arrays, == returns an element-wise array and if arr == other raises; np.array_equal is the correct comparison.

An important consideration is that mutable objects should not define __hash__ based on mutable fields, because changing them silently corrupts any set or dictionary containing the object.

17

Explain scope LEGB and nonlocal in the context of Python.

Python resolves a name by searching four scopes in order: Local (the current function), Enclosing (any outer functions, innermost first), Global (the module), and Built-in (len, print, Exception). The first match wins; if none is found, NameError is raised. This lookup diagram shows the search order for a name used inside a nested function.

Diagram: Local inner(), not found, Enclosing outer(), Global module, Built-in builtins

Scopes are decided at compile time, per function, not per block: if and for bodies do not create scopes, but comprehensions and lambdas do. Assignment makes a name local to the whole function, which produces the well-known UnboundLocalError when a function reads a global and later assigns to the same name.

def make_counter():
    count = 0
    def increment():
        nonlocal count        # rebind the enclosing variable
        count += 1
        return count
    return increment

counter = make_counter()
counter(); print(counter())   # 2

nonlocal declares that a name belongs to the nearest enclosing function scope, allowing rebinding; without it, count += 1 would create a new local and fail. global does the same for module scope. Neither is needed to mutate an object in place (items.append(x)), only to rebind the name. Class bodies are a special case: their names are not visible to methods, which must use self. or ClassName..

18

In a real project using Python, how would you apply, test, or troubleshoot scope LEGB and nonlocal?

Apply. In production code I treat scope rules as a reason to keep state explicit. Module-level globals are limited to constants and lazily initialised singletons such as a connection pool, accessed through a function rather than rebound with global. nonlocal appears in exactly one pattern I consider idiomatic: small stateful closures created by factory functions, for example a debounce helper that remembers the last call time, or an accumulator passed as a callback to a streaming parser. Anything with more than one or two pieces of state becomes a class, because self.count is more discoverable than a nonlocal three levels up.

Test. Closures created by factories are tested through their public behaviour: create two counters and verify they are independent, which proves the state lives in the closure and not in a module global. To catch accidental global mutation, a pytest fixture snapshots module attributes before a test and compares after it, failing when a test leaks state. Linters do the structural checking: ruff ’s PLW0603 warns on global statements and F823 / F841 catch unused or shadowed locals.

Troubleshoot. UnboundLocalError: cannot access local variable 'x' where it is not associated with a value (the 3.11+ wording) means the function assigns to x somewhere, usually at the bottom, turning every earlier read into a local read. The fix is to rename, pass the value in, or declare global / nonlocal deliberately. A shadowed built-in (list = fetch_rows()) produces TypeError: 'list' object is not callable later in the module; ruff rule A001 flags it. Lambdas created in a loop all seeing the final value is the late-binding problem, solved with a default argument lambda x, i=i: ... or functools.partial. Config values that “do not update” are typically modules that imported a global with from settings import DEBUG, which copied the value at import time; accessing settings.DEBUG reads the live binding.

The important consideration is that global and nonlocal are rebinding declarations, not sharing mechanisms; shared mutable objects already work without them.

19

Explain GIL and concurrency in the context of Python.

The Global Interpreter Lock is a mutex in CPython that allows only one thread to execute Python bytecode at a time. It exists because CPython’s memory management (reference counting) is not thread-safe, and a single lock is simpler and faster for single-threaded code than fine-grained locking on every object. Threads still take turns: the interpreter forces a switch roughly every 5 ms (sys.setswitchinterval), and a thread releases the GIL voluntarily whenever it blocks on I/O or enters C code that explicitly drops it, such as NumPy, hashlib and zlib. The diagram shows the practical consequence.

Diagram: Thread 1 bytecode, Thread 2 waiting, Thread 3 blocked on I/O, holds, wants, released, GIL

Consequences for concurrency design:

  • I/O-bound work (HTTP calls, database queries, file reads) scales well with threads because the GIL is released while waiting.
  • CPU-bound pure-Python work does not get faster with threads; it needs multiprocessing, a C extension that releases the GIL, or the free-threaded build.
  • Python 3.13 ships an experimental free-threaded build (PEP 703), installed as python3.13t, in which the GIL is disabled by default and sys._is_gil_enabled() reports False. It costs some single-threaded performance and many C extensions were still being ported in 3.13, so the default build keeps the GIL.
  • Python 3.12 (PEP 684) gave each sub-interpreter its own GIL, but that is exposed only through the C API in 3.12 and 3.13.

The GIL does not make Python code thread-safe: counter += 1 is several bytecodes and can still interleave, so shared mutable state needs a threading.Lock regardless.

20

In a real project using Python, how would you apply, test, or troubleshoot GIL and concurrency?

Apply. In a document-ingestion service the work splits along the GIL boundary. Downloading files and calling the OCR vendor’s API are I/O-bound, so a ThreadPoolExecutor with 32 workers handles them and throughput scales almost linearly. Text normalisation in pure Python is CPU-bound, so it runs in a ProcessPoolExecutor sized to os.cpu_count(). Image resizing stays on threads because Pillow releases the GIL inside its C code. Shared counters and caches are protected with threading.Lock, and queues between stages are queue.Queue, which is thread-safe by design.

Test. Concurrency tests target invariants, not timing. A stress test runs 50 threads incrementing a shared counter 10,000 times each and asserts the exact final total, which fails deterministically often enough to catch a missing lock. Throughput assumptions are checked with a benchmark that runs the CPU stage with 1, 2 and 4 threads and asserts no speed-up is expected, documenting why processes are used. Deadlock-prone code runs under pytest-timeout so a hang fails instead of blocking CI.

Troubleshoot. A CPU-bound thread pool that shows 100% of one core and idle others is the GIL at work; py-spy top shows all threads but only one with Python frames running, and py-spy dump confirms the rest are waiting for the GIL. The remedy is processes or a vectorised library. The opposite surprise is an I/O pool that stops scaling: usually one C library that does not release the GIL, or a requests.Session shared without a lock. Lost updates in a counter under load are the missing-lock bug. When evaluating the 3.13 free-threaded build I check sys._is_gil_enabled() at startup, because importing an extension that is not free-threading-safe re-enables the GIL silently unless PYTHON_GIL=0 is set.

The important consideration is that the GIL protects the interpreter, not your data; every shared mutable structure still needs explicit synchronisation.

21

Explain multithreading versus multiprocessing in the context of Python.

Both run work concurrently, but threads share one interpreter and its GIL, while processes each get their own interpreter, memory space and GIL.

Aspectthreadingmultiprocessing
Parallel CPU executionNo (GIL), except in C codeYes, one core per process
MemoryShared; cheap to pass objectsSeparate; data is pickled across
Start costMicrosecondsMilliseconds and a new interpreter
Failure isolationA crash kills the processA crashed worker is restartable
Best forI/O-bound: network, disk, databasesCPU-bound: parsing, numeric, compression
concurrent.futures gives both the same interface, so the choice is one line.
from concurrent.futures import ThreadPoolExecutor, ProcessPoolExecutor

def checksum(path):                        # CPU-bound
    import hashlib
    with open(path, "rb") as f:
        return hashlib.file_digest(f, "sha256").hexdigest()

with ProcessPoolExecutor() as pool:        # swap to ThreadPoolExecutor
    digests = list(pool.map(checksum, paths))   # for I/O-bound work

Processes come with constraints: arguments and results must be picklable, so lambdas, open connections and locks cannot cross the boundary, and the start method matters. On Linux the default in 3.12 and 3.13 is still fork, which copies the parent including any held locks, so 3.12 warns when forking a multi-threaded process; macOS and Windows default to spawn, which re-imports the main module and requires the if __name__ == "__main__": guard. Sharing large arrays efficiently uses multiprocessing.shared_memory instead of pickling. The 3.13 free-threaded build narrows the gap by letting threads run bytecode in parallel, but in the default build this table still holds.

22

In a real project using Python, how would you apply, test, or troubleshoot multithreading versus multiprocessing?

Apply. In a nightly report generator the fetch phase pulls from a dozen APIs and databases, so it uses a ThreadPoolExecutor; the render phase turns each dataset into PDFs with a pure-Python library and is CPU-heavy, so it runs in a ProcessPoolExecutor with max_workers=os.cpu_count() - 1 to leave a core for the scheduler. Work units are small, picklable dataclasses rather than ORM objects, and each worker process opens its own database connection in an initializer (ProcessPoolExecutor(initializer=connect)) because connections cannot be forked safely. On Linux I set the start method to spawn or forkserver explicitly to avoid inheriting the parent’s threads and locks.

Test. Unit tests run the worker function directly, single-threaded, with the pool replaced by a fake executor that calls fn synchronously; this keeps tests fast and deterministic. A separate integration test runs the real pool with two workers to verify that tasks are picklable, that the __main__ guard is present, and that results come back in the expected order via executor.map. Resource tests assert that the pool is shut down (with block) and that no worker processes survive, using multiprocessing.active_children().

Troubleshoot. PicklingError: Can't pickle <function <lambda>> or local object means a closure or nested function was submitted to a process pool; move it to module level. A process pool that hangs on startup under macOS or in a Docker container is almost always a missing if __name__ == "__main__": guard with the spawn method, visible as recursive re-imports in the log. Deadlocks after fork show up as children stuck inside the logging module; the fix is forkserver / spawn or os.register_at_fork. For threads, py-spy dump reveals a pool where every worker waits on the same lock, and a thread pool with no speed-up on a CPU task is the GIL rather than a bug.

The important consideration is that processes multiply memory usage: a 2 GB in-memory model times eight workers will not fit, so load it in the initializer once per worker or share it via shared_memory.

23

Explain asyncio and async await in the context of Python.

asyncio provides single-threaded cooperative concurrency: one event loop runs many coroutines, and each coroutine gives up control at an await whenever it would otherwise block on I/O. async def defines a coroutine function; calling it returns a coroutine object that does nothing until the loop drives it. await suspends the current coroutine until the awaited object completes, letting the loop run others meanwhile. The diagram shows that cycle.

Diagram: Task A await socket, waiting on fd, selector epoll / kqueue, fd ready, Event loop, run until await, next, Task B running, Task C ready
import asyncio, httpx

async def fetch(client, url):
    r = await client.get(url, timeout=5)
    return r.status_code

async def main():
    async with httpx.AsyncClient() as client:
        async with asyncio.TaskGroup() as tg:       # 3.11+
            tasks = [tg.create_task(fetch(client, u)) for u in urls]
    print([t.result() for t in tasks])

asyncio.run(main())

Essentials:

  • Only asynchronous libraries yield to the loop. A blocking call such as time.sleep or requests.get inside a coroutine freezes every other task; use await asyncio.sleep, an async client, or await asyncio.to_thread(blocking_fn).
  • asyncio.run() creates the loop, runs the coroutine and closes it. TaskGroup (3.11) and asyncio.timeout() (3.11) replace gather and wait_for for structured cancellation and error handling.
  • Coroutines do not run until scheduled: a forgotten await produces a RuntimeWarning: coroutine ... was never awaited.
  • asyncio is a win for thousands of concurrent connections; it does not speed up CPU-bound code, which still needs processes.
24

In a real project using Python, how would you apply, test, or troubleshoot asyncio and async await?

Apply. A notification gateway I worked on fans each event out to e-mail, SMS and push providers. With asyncio the handler awaits all three in a TaskGroup, so latency is the slowest provider rather than the sum, and a single worker holds thousands of in-flight requests. Database access goes through an async driver (asyncpg), HTTP through httpx.AsyncClient with a shared connection pool, and the one legacy SDK that is synchronous is isolated behind asyncio.to_thread. Every outbound call sits inside async with asyncio.timeout(3):, and a Semaphore caps concurrency per provider to respect their rate limits.

Test. Tests are written with pytest-asyncio (or anyio), marking coroutine tests and using asyncio.Event and fake clients instead of sleeping for timing. respx or httpx.MockTransport stubs the HTTP layer so the full fan-out path runs without the network. Cancellation and timeout paths get explicit tests: a provider that never responds must raise TimeoutError within the budget and must not leave tasks running, which I assert with asyncio.all_tasks() after the test. Running the suite with PYTHONASYNCIODEBUG=1 turns forgotten awaits into logged errors.

Troubleshoot. Rising latency across unrelated endpoints is the signature of a blocking call in the loop; debug mode logs Executing <Task ...> took 2.3 seconds, and py-spy dump shows the loop thread inside a synchronous driver or json.loads on a huge payload. The fix is an async library or to_thread. “Task was destroyed but it is pending” at shutdown means fire-and-forget create_task calls without keeping references; TaskGroup fixes the lifecycle. RuntimeError: attached to a different loop comes from creating a client or lock at import time with a different loop than asyncio.run later uses; create them inside main(). Memory growth often turns out to be an unbounded gather over a large list, replaced with a semaphore or asyncio.Queue with workers.

The important consideration is that one blocking call anywhere stalls the whole process; asyncio is only as good as the least asynchronous library in the dependency tree.

25

Explain context managers and with in the context of Python.

A context manager is an object that defines __enter__ and __exit__, and the with statement guarantees that __exit__ runs when the block ends, whether it ends normally, by return, or by an exception. It replaces try/finally boilerplate for acquiring and releasing resources: files, locks, database transactions, temporary directories, and timing or logging around a block.

import contextlib, time

@contextlib.contextmanager
def timed(label):
    start = time.perf_counter()
    try:
        yield                                  # the with-block runs here
    finally:
        print(f"{label}: {time.perf_counter() - start:.3f}s")

with timed("load"), open("data.csv", encoding="utf-8") as f:
    rows = f.read().splitlines()
#=> prints "load: 0.012s" even if read() raised

How the protocol works:

  • __enter__ runs first and its return value is bound by as. __exit__(exc_type, exc, tb) receives the exception, if any; returning True suppresses it, anything falsy re-raises it after cleanup.
  • contextlib.contextmanager turns a generator into a context manager: code before yield is the enter phase, code after it is the exit phase, and try/finally around the yield is what makes cleanup reliable.
  • Several managers can be stacked in one with, and since Python 3.10 they may be wrapped in parentheses across lines. contextlib.ExitStack handles a variable number of managers.
  • async with uses __aenter__ / __aexit__ for asynchronous resources like database sessions and HTTP clients.

contextlib also provides suppress, redirect_stdout, closing, nullcontext and chdir (3.11) for common cases. A lock used with with lock: is the smallest example and the one most people meet first.

26

In a real project using Python, how would you apply, test, or troubleshoot context managers and with?

Apply. In a service backed by PostgreSQL, the unit of work is a context manager: with uow() as session: opens a transaction, yields the session, commits on normal exit and rolls back when an exception propagates. Request handlers cannot forget to commit or roll back because the structure of the code enforces it. The same pattern wraps temporary files (tempfile.TemporaryDirectory), feature-flag overrides in tests, metric timers, and a contextlib.ExitStack that opens a dynamic number of output files for a multi-partition export and closes all of them in reverse order even if the fifth one fails to open.

Test. A context manager is tested through its three exits: normal completion, return inside the block, and an exception. For the transaction manager a fake session records calls and the test asserts commit was called exactly once on success and rollback exactly once when the block raises, and that the original exception still propagates. I also test that __exit__ does not swallow exceptions by accident, since returning a truthy value there is a common mistake. pytest.raises is itself a context manager, which makes the exception-path test read naturally.

Troubleshoot. Connection-pool exhaustion after a few hours is almost always a resource acquired without with on an error path; ResourceWarning: unclosed <socket> appears when running with python -X dev or -W error::ResourceWarning, and tracebacks name the allocation site. A generator-based manager that fails to clean up points at a missing try/finally around the yield. “generator didn’t yield” or “didn’t stop” errors mean the generator has a conditional yield or yields twice. Exceptions that vanish silently trace back to an __exit__ returning True, or a bare except inside the cleanup. For async code, calling an async with manager with a plain with raises AttributeError: __enter__, which is the fastest way to spot a sync/async mismatch.

The important consideration is that cleanup in __exit__ must itself be exception-safe; if it raises, it replaces the original exception and hides the real cause.

27

Explain exception handling and finally in the context of Python.

Exceptions are objects derived from BaseException that unwind the call stack until a matching except clause catches them. A try statement has four optional parts: except handles a matching exception, else runs only when the try block raised nothing, and finally runs on every exit path: success, handled exception, unhandled exception, return, break or continue. That guarantee is what makes finally the right place for cleanup when no context manager exists.

def load_config(path):
    f = None
    try:
        f = open(path, encoding="utf-8")
        data = parse(f.read())
    except FileNotFoundError:
        return {}                      # finally still runs before returning
    except ValueError as exc:
        raise ConfigError(f"bad config in {path}") from exc
    else:
        return data
    finally:
        if f is not None:
            f.close()

Points that interviewers probe:

  • Catch the narrowest class that you can handle. Bare except: also catches KeyboardInterrupt and SystemExit; except Exception: is the widest sensible net.
  • raise ... from exc sets __cause__ for explicit chaining; an exception raised inside a handler without from gets an implicit __context__, shown as “During handling of the above exception, another exception occurred”.
  • A return inside finally overrides the pending return value and silently discards any in-flight exception. It is legal, almost always a bug, and Python 3.14 starts warning about it.
  • Python 3.11 added ExceptionGroup and except* for handling several concurrent failures, plus exc.add_note() to attach context without wrapping.
  • try/except costs almost nothing when no exception is raised (3.11 made it zero-cost), so the “easier to ask forgiveness than permission” style is idiomatic for dictionary access and type conversion.

Custom exceptions should form a small hierarchy rooted at one application base class so callers can catch broadly or narrowly.

28

In a real project using Python, how would you apply, test, or troubleshoot exception handling and finally?

Apply. A payment-capture service defines a hierarchy PaymentError with DeclinedError, GatewayTimeout and DuplicateCapture subclasses. The HTTP layer maps each to a status code in one place; the domain layer raises them with raise DeclinedError(code) from vendor_exc so the vendor traceback is preserved in logs without leaking into the API response. Retries happen only for the subclass that is safe to retry. finally blocks are rare because context managers handle most cleanup, but one appears in the job runner to mark a job as finished in the database whatever happened. Unexpected exceptions bubble to a single middleware that logs with logger.exception(), attaches a correlation ID with add_note, and returns a 500.

Test. Every custom exception has a test that it is raised for the intended input, using pytest.raises(DeclinedError) as info and asserting on info.value.code and on info.value.__cause__. The middleware test feeds it each exception class and asserts the status code and that the response body carries no stack trace. Cleanup paths are tested by making the body raise and asserting the finally effect, such as the job row being marked failed. A linter configuration (ruff rules BLE001, E722, TRY) rejects bare excepts and except Exception: pass.

Troubleshoot. Silent failures almost always trace back to an over-broad except Exception that logs at debug level or swallows the error; the fix is to narrow it and re-raise. “During handling of the above exception” chains with an unrelated second error usually come from cleanup code in a handler that itself fails, hiding the root cause; __context__ in the traceback shows both. A finally with return produced a function that “never fails” and masked a TypeError for weeks. ExceptionGroup tracebacks from TaskGroup confused on-call engineers until the handler used except* to split retryable from fatal sub-exceptions.

The important consideration is that an exception handler should either handle the error fully or add information and re-raise; doing neither destroys the evidence needed to fix the problem.

29

Explain classes inheritance and MRO in the context of Python.

A class is a namespace that produces instances; inheritance lets a class reuse and specialise another by listing bases in its definition. Python supports multiple inheritance, so it needs a deterministic rule for which implementation of a method is found first: the Method Resolution Order, computed with the C3 linearisation algorithm and exposed as Cls.__mro__. C3 guarantees that a class precedes its bases and that the order in the base list is respected, and it raises TypeError if those constraints cannot be satisfied. This hierarchy shows the classic diamond.

Diagram: ApiClient, LoggingMixin, RetryMixin, Base, object
class Base:
    def send(self, req): return f"sent {req}"

class LoggingMixin(Base):
    def send(self, req):
        print("log:", req)
        return super().send(req)

class RetryMixin(Base):
    def send(self, req):
        return super().send(req) or self.send(req)

class ApiClient(LoggingMixin, RetryMixin): pass

print([c.__name__ for c in ApiClient.__mro__])
#=> ['ApiClient', 'LoggingMixin', 'RetryMixin', 'Base', 'object']

super() does not mean “my parent”; it means “the next class after me in the MRO of the instance’s type”. That is what lets each mixin call super().send() and still reach RetryMixin before Base, a pattern known as cooperative multiple inheritance. For it to work, every class in the chain must call super() and accept compatible arguments.

Attribute lookup follows the same order for data attributes, and isinstance / issubclass consult it too. Abstract base classes (abc.ABC) and Protocol classes shape hierarchies without forcing shared implementation.

30

In a real project using Python, how would you apply, test, or troubleshoot classes inheritance and MRO?

Apply. In an integrations platform each external provider is a subclass of an abstract Connector with fetch(), map() and push() declared as abstractmethods, so a missing method fails at instantiation instead of at 3 a.m. Cross-cutting behaviour is added by small mixins (RateLimitedMixin, CachedMixin) that each override one method and call super(), so a concrete connector is class Shopify(CachedMixin, RateLimitedMixin, Connector) and the MRO is the execution order of those layers. Hierarchies stay shallow, two or three levels, and composition replaces inheritance when the relationship is “has a” rather than “is a”.

Test. The abstract base gets a test that instantiating it, or a subclass missing a method, raises TypeError. For mixin stacks I assert on Cls.__mro__ directly in a test, because the order is part of the design: if someone reorders bases, the test explains why that matters. Behaviour tests stub Connector.push and assert that the cache layer is consulted before the rate limiter. typing.override (3.12) on every overriding method lets mypy or pyright fail the build when a base-class method is renamed and an override silently becomes a dead method.

Troubleshoot. TypeError: Cannot create a consistent method resolution order (MRO) appears when base classes are listed in an order that contradicts their own hierarchy, typically class X(Base, Mixin) where Mixin already subclasses Base; swapping the order fixes it. A mixin whose code never runs is usually placed after a class that does not call super(), breaking the chain; Cls.__mro__ and a breakpoint confirm it. TypeError: __init__() got an unexpected keyword argument in a cooperative hierarchy means one __init__ does not accept and forward **kwargs. Mutable class attributes (items = []) shared by every instance produce cross-instance state leaks; the fix is to assign them in __init__.

The important consideration is that multiple inheritance only stays manageable when every participating class is written cooperatively and does one thing.

31

Explain classmethod staticmethod property in the context of Python.

All three are built-in descriptors that change how a function defined in a class body is bound when accessed through the class or an instance.

DecoratorFirst argumentTypical use
plain methodself (the instance)Behaviour that needs instance state
@classmethodcls (the class)Alternative constructors; factories that respect subclasses
@staticmethodnoneUtility grouped in the class for namespacing
@propertyself, called on attribute accessComputed or validated attributes with field-like syntax
from datetime import date

class Invoice:
    def __init__(self, number: str, issued: date):
        self.number, self.issued = number, issued

    @classmethod
    def from_row(cls, row: dict) -> "Invoice":
        return cls(row["number"], date.fromisoformat(row["issued"]))

    @staticmethod
    def is_valid_number(value: str) -> bool:
        return value.startswith("INV-")

    @property
    def age_days(self) -> int:
        return (date.today() - self.issued).days

A classmethod receives the class it was called on, so SubInvoice.from_row(...) builds a SubInvoice without any change. A staticmethod is just a function living in the class namespace; it signals that the logic needs neither instance nor class. A property turns inv.age_days into a method call, with optional @age_days.setter and .deleter for validation on assignment, which lets a plain attribute become computed later without breaking callers. functools.cached_property computes once per instance and stores the result.

One version detail: stacking @classmethod on top of @property to make a “class property” worked from 3.9, was deprecated in 3.11 and no longer works in 3.13, where Cls.attr returns a bound method instead of the computed value; a metaclass property or a plain classmethod is the alternative.

32

In a real project using Python, how would you apply, test, or troubleshoot classmethod staticmethod property?

Apply. Domain models in an inventory service use classmethods as named constructors: StockLevel.from_event(evt), StockLevel.from_db_row(row) and StockLevel.empty(sku), which keeps __init__ simple and makes every creation path searchable. Properties expose derived data, such as available = on_hand - reserved, and validate writes: the reorder_point setter rejects negatives, so bad data cannot enter through attribute assignment. Expensive derived values that do not change over the object’s life, like a parsed configuration tree, use functools.cached_property. Staticmethods are limited to pure helpers that read better next to the class, for example Money.parse("12.50 EUR").

Test. Alternative constructors get a test per input shape, including a subclass test asserting type(Sub.from_row(r)) is Sub, which is the whole point of cls. Property setters get boundary tests (pytest.raises(ValueError) for -1). For cached_property I test that the underlying computation runs once by patching it with a Mock and asserting call_count == 1 after two accesses, and I test that cache invalidation (del obj.attr) works where the design relies on it.

Troubleshoot. TypeError: 'int' object is not callable after accessing obj.total() means someone converted total into a property and a caller still has parentheses; the reverse produces a bound method where a value was expected, visible as <bound method ...> in logs. A property that hides an expensive database call becomes a performance incident when a template loops over it; cProfile shows the property at the top, and the fix is cached_property or an explicit method name that signals cost. cached_property fails with TypeError: No '__dict__' attribute on classes using __slots__, so those need a manual cache slot. Mock-based tests that try to patch a property with patch.object(obj, "prop") need new_callable=PropertyMock or patching on the class.

The important consideration is that a property should behave like an attribute: cheap, side-effect free and unlikely to raise, otherwise a method is the honest interface.

33

Explain dunder methods in the context of Python.

Dunder (“double underscore”) methods such as __init__, __len__ and __eq__ are the hooks through which Python’s syntax and built-in functions talk to objects. Writing len(x) calls x.__len__(), a + b calls a.__add__(b) (falling back to b.__radd__(a)), for item in x calls x.__iter__(), and with x: calls __enter__ / __exit__. Implementing the right ones makes a user-defined class feel like a built-in type.

from dataclasses import dataclass

@dataclass(frozen=True)
class Money:
    amount: int                      # minor units
    currency: str

    def __add__(self, other):
        if not isinstance(other, Money) or other.currency != self.currency:
            return NotImplemented    # lets Python try __radd__ or raise
        return Money(self.amount + other.amount, self.currency)

    def __str__(self):
        return f"{self.amount / 100:.2f} {self.currency}"

print(Money(1050, "EUR") + Money(250, "EUR"))   # 13.00 EUR

Groups worth knowing:

  • Construction and representation: __new__, __init__, __repr__ (unambiguous, for developers), __str__ (for users), __format__.
  • Comparison and hashing: __eq__, __lt__ and friends (functools.total_ordering fills in the rest), __hash__, __bool__.
  • Containers: __len__, __getitem__, __setitem__, __contains__, __iter__, __next__.
  • Callables and attributes: __call__, __getattr__ (fallback only), __getattribute__ (every access), __setattr__, __slots__.
  • Lifecycle and protocols: __enter__ / __exit__, __aenter__ / __aexit__, __await__, __del__ (unreliable, avoid), __init_subclass__, __class_getitem__.

Returning NotImplemented from binary operators, rather than raising, is what allows mixed-type arithmetic to work. Dunder names are reserved for the language; inventing new ones is legal but conflicts with future versions.

34

In a real project using Python, how would you apply, test, or troubleshoot dunder methods?

Apply. In a scheduling system TimeRange implements __contains__ so if ts in shift: reads naturally, __and__ to compute overlaps, __lt__ so ranges sort by start time, and __repr__ that prints the ISO bounds, which makes log lines and failed-test output self-explanatory. A FeatureFlags object implements __getattr__ to look up flags lazily and __dir__ so tab completion works. A Pipeline class implements __call__ so it can be passed wherever a plain function is expected, and __iter__ so its stages can be inspected. __slots__ on the millions of Event instances in the ingestion path cut memory by roughly half and blocked typo’d attribute assignments.

Test. Each operator gets tests for the happy path, the reflected form (other + money when other is a foreign type returning NotImplemented), and the mixed-currency failure which must raise TypeError. __repr__ is tested with eval(repr(obj)) == obj where that round-trip is a design goal. For __eq__ / __hash__ pairs, a set-insertion test guards the contract. Protocol conformance is checked structurally: isinstance(obj, collections.abc.Sequence) or a Protocol type checked by mypy.

Troubleshoot. __getattr__ that raises anything other than AttributeError breaks hasattr, getattr(obj, name, default) and copy.deepcopy, producing bizarre errors far from the class; the fix is to raise AttributeError for unknown names. An infinite recursion inside __getattribute__ or __setattr__ comes from accessing self.x instead of object.__getattribute__(self, "x"). Objects that are “truthy when empty” are missing __bool__ or __len__. Pickling failures after adding __slots__ need __getstate__ / __setstate__. A __del__ that touches a database connection fails randomly at interpreter shutdown; replacing it with a context manager or weakref.finalize resolves it.

The important consideration is that dunder methods make syntax do work invisibly, so each should be cheap, predictable, and consistent with its built-in counterpart’s contract.

35

Explain dataclasses and typing in the context of Python.

dataclasses generates boilerplate for classes that mainly hold data, and typing provides the annotations that static checkers such as mypy and pyright use to find bugs before run time. They fit together because @dataclass reads the class’s annotations to decide which fields exist.

from dataclasses import dataclass, field
from typing import Literal, Self

@dataclass(slots=True, kw_only=True)
class Order:
    id: int
    status: Literal["new", "paid", "shipped"] = "new"
    lines: list["Line"] = field(default_factory=list)
    total_cents: int = field(default=0, compare=False)

    def with_status(self, status: Literal["paid", "shipped"]) -> Self:
        return dataclasses.replace(self, status=status)

What @dataclass generates and controls:

  • __init__, __repr__ and __eq__ from the fields; order=True adds ordering methods, frozen=True makes instances immutable and hashable, slots=True (3.10) removes the per-instance dictionary, and kw_only=True (3.10) forces keyword construction.
  • field() handles mutable defaults via default_factory, excludes fields from comparison or repr, and __post_init__ runs validation.
  • dataclasses.asdict, astuple and replace cover serialisation and copy-with-change.

Typing essentials in 3.12 and 3.13: built-in generics (list[int], dict[str, Order]), unions with |, Optional written as X | None, TypedDict for dictionary shapes, Protocol for structural typing, Self, Literal, and the 3.12 type Alias = ... statement and class Stack[T]: generic syntax from PEP 695. Python 3.13 added TypeIs, ReadOnly for TypedDict fields and defaults for type parameters. Annotations are not enforced at run time; a dataclass happily accepts a string for an int field, which is why pydantic or attrs with validators exist for untrusted input.

36

In a real project using Python, how would you apply, test, or troubleshoot dataclasses and typing?

Apply. Internal domain objects in an order service are frozen, slotted dataclasses: they are cheap, hashable, and impossible to mutate by accident, and replace() gives the functional update style. Boundary data (HTTP bodies, queue messages) goes through pydantic models instead, because there the types must be enforced, then converted to dataclasses for the domain layer. Type hints are complete across the codebase, mypy --strict runs in CI, and Protocol classes define what a repository or a payment gateway must offer, so test doubles need no inheritance. Literal types for status fields and TypedDict for the few legacy dictionary payloads catch most of the “wrong string” bugs at check time.

Test. The type checker is the first test; CI fails on new errors and warn_unused_ignores keeps the # type: ignore count from growing. Dataclass behaviour tests cover __post_init__ validation (pytest.raises(ValueError)), equality ignoring compare=False fields, and that frozen instances raise FrozenInstanceError on assignment. For serialisation, a round-trip test asserts from_dict(asdict(obj)) == obj. Where a generic function must accept a Protocol, a small test instantiates a minimal conforming class and passes it, confirming the protocol is not over-specified.

Troubleshoot. ValueError: mutable default <class 'list'> for field means a list or dict default without default_factory. TypeError: non-default argument follows default argument is solved with kw_only=True or reordering. Inheritance with slots=True fails if the base class is not slotted too, and cached_property needs a __dict__, so slotted dataclasses must not use it. mypy reporting Incompatible types on a dataclass field usually points at Optional fields initialised to None without X | None in the annotation. Circular references between typed modules are resolved with from __future__ import annotations or TYPE_CHECKING imports. Run-time NameError for a forward-referenced type means code called typing.get_type_hints() on a module where the name is only imported under TYPE_CHECKING.

The important consideration is that annotations are documentation checked by tools, not run-time guarantees; validation at system boundaries is still required.

37

Explain memory management and garbage collection in the context of Python.

CPython manages memory with two cooperating mechanisms: reference counting as the primary collector and a cyclic garbage collector for what reference counting cannot handle. Every object carries a count of references to it; when the count drops to zero the object is freed immediately and deterministically. Cycles (a.ref = b; b.ref = a) keep counts above zero forever, so the gc module periodically scans container objects to find unreachable cycles. The diagram shows where each piece of the system sits.

Diagram: Python objects, refcount = 0, cycles, Reference counting, Cyclic GC 3 generations, free, pymalloc small-object pools, arenas, …

Key behaviour in 3.12 and 3.13:

  • The cyclic collector is generational: new container objects start in generation 0 and survivors are promoted; collection of generation 0 is triggered when allocations minus deallocations exceed a threshold (gc.get_threshold() returns (700, 10, 10) in 3.12 and (2000, 10, 10) in 3.13). It can be tuned, disabled with gc.disable(), or run on demand with gc.collect().
  • Objects with __del__ in cycles are collected since 3.4 (PEP 442), but finaliser order is undefined.
  • Small objects (up to 512 bytes) come from pymalloc pools inside 256 KiB arenas (1 MiB since 3.10 on 64-bit builds); memory is returned to the OS only when a whole arena is empty, so a process’s RSS rarely shrinks after a spike.
  • sys.getrefcount(x) reports the count (one higher because of the argument), tracemalloc tracks allocations by source line, and weakref lets caches reference objects without keeping them alive.

Memory is therefore freed promptly in the common case, and “leaks” in Python are almost always live references held by caches, globals or closures rather than collector failures.

38

In a real project using Python, how would you apply, test, or troubleshoot memory management and garbage collection?

Apply. In a long-running worker that processes millions of small events, memory design is mostly about not keeping references: results are streamed with generators, caches are bounded (functools.lru_cache(maxsize=10_000) or cachetools.TTLCache), and observer registrations use weakref.WeakSet so an unsubscribed listener does not survive. Hot-path objects use __slots__ or slotted dataclasses, which roughly halves per-instance memory. On a web server that forks workers after loading a large model, gc.freeze() before forking stops the collector from dirtying shared pages, and batch jobs that build huge object graphs disable the collector during the build and call gc.collect() once at the end.

Test. Memory regressions are tested with tracemalloc: take a snapshot, run the operation 1,000 times, take another, and assert the growth of the top statistics is below a budget. A simpler guard asserts that a weakref.ref to a request object is dead after the handler returns. These tests run in CI on a representative fixture, not the full dataset.

Troubleshoot. RSS growing linearly in production is the common page. First I confirm whether Python objects are growing (tracemalloc snapshot diff or objgraph.show_most_common_types()) or whether it is native memory from a C extension, which tracemalloc cannot see. Typical culprits: an unbounded dictionary cache keyed by request ID, logging handlers appended on every request, exception objects stored in a list (each one holds its traceback and frames), and lru_cache on a method, which keeps every self alive. A “leak” that is really fragmentation shows RSS staying high while tracemalloc is flat; the remedy is recycling workers (max_requests in gunicorn). Long GC pauses in a latency-sensitive service appear as periodic spikes; gc.set_debug(gc.DEBUG_STATS) measures them and reducing live container objects fixes them.

The important consideration is that gc.collect() is not a fix for a leak caused by a live reference; it only frees what is already unreachable.

39

Explain virtual environments and packaging in the context of Python.

A virtual environment is an isolated directory with its own site-packages and a python launcher that points to a base interpreter. Packages installed inside it do not affect the system Python or other projects, which is what makes two services with conflicting dependency versions coexist on one machine. python -m venv .venv creates one; activating it only adjusts PATH, and .venv/bin/python -m pip ... works without activation. Tools such as uv, poetry, hatch and pipenv create and manage the same kind of environment with extra conveniences like lock files.

Packaging is the process of describing a project so it can be built and installed. Modern projects declare everything in pyproject.toml: metadata and dependencies in the [project] table (PEP 621), and the build backend in [build-system]. The diagram shows the path from source to an installed package.

Diagram: pyproject.toml + src/, python -m build, build backend hatchling / setuptools, sdist .tar.gz, wheel .whl, twine upload, PyPI / private index, …
[project]
name = "order-service"
version = "1.4.0"
requires-python = ">=3.12"
dependencies = ["httpx>=0.27", "pydantic>=2,<3"]

[project.optional-dependencies]
dev = ["pytest", "mypy", "ruff"]

[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

Wheels are pre-built zip archives installed by copying; sdists must be built on the target machine. pip install -e . installs a project in editable mode for development. Version pins for applications belong in a lock file (requirements.txt from pip-compile, uv.lock, or the standardised pylock.toml from PEP 751); libraries declare ranges instead. distutils was removed in Python 3.12, so legacy setup.py projects must depend on setuptools explicitly.

40

In a real project using Python, how would you apply, test, or troubleshoot virtual environments and packaging?

Apply. Every service repository has a pyproject.toml with a src/ layout, a pinned requires-python, and ranges for direct dependencies only. A lock file from uv lock (or pip-compile) pins the full transitive tree with hashes, and that lock, not the ranges, is what CI and the Dockerfile install, so a build today and a build next month are identical. Developers create the environment with uv sync or python -m venv .venv && pip install -e ".[dev]". Shared internal libraries are built as wheels with python -m build, published to a private index and consumed through the same lock. Docker images use multi-stage builds: dependencies go into a venv in the builder stage and are copied into a slim runtime image.

Test. CI runs the suite in a fresh environment created from the lock file, which proves the lock is complete, across a matrix of the supported interpreters (3.12 and 3.13) because requires-python is a promise. For libraries, a job installs the built wheel into an empty venv and imports the package, catching missing package data that editable installs hide. pip check verifies the resolved set has no conflicts, and a scheduled job attempts a dependency upgrade and runs the tests to surface breakage early.

Troubleshoot. “Works on my machine” almost always means something was installed outside the lock; pip freeze diffed against the lock reveals it. ModuleNotFoundError inside Docker while tests pass locally points to the src/ package missing from the wheel or to the system interpreter being used instead of the venv; python -c "import sys; print(sys.prefix)" settles it. ResolutionImpossible is diagnosed by reading the two requirements it names and loosening the stricter one. A compiled dependency failing to install on a new interpreter means no wheel exists yet for that version; staying on the previous interpreter until upstream ships one is the pragmatic answer.

The important consideration is that lock files belong to applications; a library that pins exact versions makes itself uninstallable alongside anything else.

41

Explain modules imports and circular imports in the context of Python.

A module is a .py file (or a C extension) whose top-level code runs once, on first import, to populate a module object that is then cached in sys.modules. A package is a directory of modules, with or without __init__.py (namespace packages, PEP 420). import a.b finds the module through sys.path, executes it top to bottom, and binds the name; from a.b import name additionally looks up name in the finished module. Subsequent imports anywhere in the process return the cached object, so module state is effectively a process-wide singleton.

A circular import occurs when module a imports b while b is itself in the middle of importing a. Because a is already registered in sys.modules but only partially executed, import a inside b succeeds, yet from a import something fails if something has not been defined yet. The diagram shows the failing sequence.

Diagram: models.py line 1: import services, executes, services.py line 1: from models import Order, sys.modules hit, models.py partially loaded, …

Ways to break the cycle, in order of preference:

  • Move the shared definitions into a third module both can import, or merge the two if they are that entangled.
  • Import the module rather than the name (import models and then models.Order at call time), so the attribute is resolved after both modules finish loading.
  • Import inside the function that needs it, deferring the import to call time.
  • For imports needed only by type annotations, guard them with if TYPE_CHECKING: and use string annotations or from __future__ import annotations.

Python 3.8+ adds “(most likely due to a circular import)” to the ImportError message, and python -X importtime prints the import tree with timings, which also shows the order in which the cycle was entered.

42

In a real project using Python, how would you apply, test, or troubleshoot modules imports and circular imports?

Apply. Package layout follows dependency direction: domain imports nothing from the application, services imports domain, api imports services, and nothing imports upward. Enforcing this with a tool (import-linter contracts in CI) is cheaper than untangling cycles later. Module-level code is kept to definitions; anything that connects to a database or reads configuration happens inside functions, so importing a module is always side-effect free and fast. Public API of a package is re-exported in __init__.py with __all__, while heavy optional dependencies (pandas, boto3) are imported lazily inside the functions that need them to keep CLI start-up quick.

Test. A smoke test imports every module in the package (pkgutil.walk_packages) in a fresh interpreter, which catches circular imports and import-time side effects regardless of which module the test suite happened to import first. The layering contracts run as a CI step. Start-up time is tracked by running python -X importtime -c "import myservice" in CI and failing if the total exceeds a budget, which has caught a stray import torch at module level more than once.

Troubleshoot. ImportError: cannot import name 'X' from partially initialized module is the circular import itself; -X importtime or the traceback shows which module was mid-execution. The quick fix is a local import; the real fix is moving the shared type to a leaf module. AttributeError: module 'x' has no attribute 'y' with a correct-looking module usually means a file in the project shadows a standard-library or third-party name (logging.py, test.py); print(x.__file__) exposes it. Tests that pass alone but fail together are import-order dependent, typically because one module monkeypatches another at import. Behaviour that differs between python script.py and python -m pkg.script comes from sys.path[0] being the script’s directory in the first case, so relative imports break; running as a module is the fix.

The important consideration is that a circular import is a design signal, two modules that know too much about each other, and a local import only hides the symptom.

43

Explain lambda map filter reduce in the context of Python.

These are the functional-programming primitives: lambda creates a small anonymous function, map applies a function to every element, filter keeps elements for which a predicate is true, and reduce folds a sequence into a single value by repeatedly combining an accumulator with the next element. In Python 3, map and filter return lazy iterators, and reduce lives in functools.

from functools import reduce

prices = [19.99, 5.0, 120.0, 42.5]

with_tax = map(lambda p: round(p * 1.2, 2), prices)   # lazy
large = filter(lambda p: p > 20, prices)              # lazy
total = reduce(lambda acc, p: acc + p, prices, 0.0)   # eager

print(list(with_tax))        # [23.99, 6.0, 144.0, 51.0]
print(list(large))           # [120.0, 42.5]
print(total)                 # 187.49

reduce(f, items, initial) computes f(f(f(initial, items[0]), items[1]), ...). Without initial the first element seeds the accumulator and an empty input raises TypeError, so the initial value is worth supplying.

Idiomatic Python usually prefers comprehensions and built-ins to these functions: [p * 1.2 for p in prices] reads more clearly than map with a lambda, [p for p in prices if p > 20] replaces filter, and sum, min, max, any, all, math.prod and "".join cover almost every real use of reduce. map still earns its place when the function already exists (map(str, numbers), map(int, line.split())) or when applying a function in parallel through executor.map. operator.add, operator.itemgetter and operator.attrgetter replace many lambdas with faster, named callables.

44

In a real project using Python, how would you apply, test, or troubleshoot lambda map filter reduce?

Apply. In a data-cleaning pipeline these primitives appear at the edges rather than in the core logic. map(str.strip, raw_lines) and map(int, ids) convert input cheaply and lazily. executor.map(process_file, paths) fans work out to a thread or process pool and pool.imap_unordered streams results back in completion order. reduce is reserved for genuinely associative folds: merging a list of dictionaries with reduce(operator.or_, dicts, {}) or composing a list of transformation functions into one callable. Lambdas stay limited to key= arguments; anything with branching gets a named function, and operator.itemgetter("sku") replaces lambda r: r["sku"].

Test. Named functions passed to map and filter are unit-tested directly with table-driven cases, which is the main reason to prefer them over inline lambdas. Pipeline functions returning iterators are tested with list(...) on small inputs, plus a test with an empty input to confirm reduce has a correct initial value. For executor.map paths a test verifies that results preserve input order (map) or that all results arrive (imap_unordered), and that an exception in one item surfaces when its result is consumed, not silently.

Troubleshoot. The most common production bug is consuming a map or filter object twice: the second pass is empty, so a report shows zero rows after the first validation step iterated it. Converting to a list at the point where reuse is needed fixes it. TypeError: reduce() of empty iterable with no initial value appears on the first empty batch in production after months of non-empty test data. Lambdas in a multiprocessing pool raise PicklingError; a module-level function or functools.partial resolves it. A lambda defined in a loop capturing the loop variable produces every callback seeing the last value, fixed with a default argument.

The important consideration is that map, filter and reduce are tools for applying existing functions, not an excuse to write logic inside lambdas that no test can target.

45

Explain file handling and encoding in the context of Python.

open(path, mode, encoding=...) returns a file object, and the mode decides whether it is a text file that yields str or a binary file that yields bytes. Text mode decodes on read and encodes on write using the encoding argument; binary mode ("rb", "wb") hands over raw bytes untouched. The with statement closes the file deterministically, including on exceptions.

from pathlib import Path

log = Path("app.log")
with log.open("a", encoding="utf-8") as f:          # text, append
    f.write("started\n")

for line in log.open(encoding="utf-8"):             # streams line by line
    print(line.rstrip("\n"))

raw: bytes = Path("image.png").read_bytes()         # binary helper
text = Path("legacy.csv").read_text(encoding="cp1252", errors="replace")

Encoding rules that matter:

  • Always pass encoding explicitly for text files. The default is locale.getencoding(), which is UTF-8 on most Linux and macOS systems but a legacy code page on many Windows machines, so the same code reads a file correctly on one platform and raises UnicodeDecodeError on another. Running with -X warn_default_encoding (3.10+) emits an EncodingWarning wherever the default is relied on, and UTF-8 mode (-X utf8 or PYTHONUTF8=1) forces UTF-8 everywhere.
  • errors= controls undecodable bytes: "strict" (default) raises, "replace" inserts U+FFFD, "ignore" drops them, and "surrogateescape" round-trips them, which is what the OS file-system APIs use.
  • "utf-8-sig" consumes a byte-order mark on read and writes one, which Excel-generated CSVs often need.
  • newline="" must be passed when writing CSV with the csv module, otherwise Windows gets doubled line endings.

pathlib.Path is the modern API for paths and offers read_text, write_text, read_bytes, iterdir and glob. For large files, iterate line by line or read fixed-size chunks rather than calling read(), and use shutil.copyfileobj for copying streams.

46

In a real project using Python, how would you apply, test, or troubleshoot file handling and encoding?

Apply. A bank-statement import service receives CSV and fixed-width files from dozens of institutions, each with its own encoding. The ingest layer never opens files in text mode blindly: it reads the first few kilobytes as bytes, checks for a BOM, applies the encoding declared in the institution’s profile, and falls back to charset-normalizer detection only as a last resort, logging what it chose. Files are streamed line by line, and output is written to a temporary file and renamed with os.replace so a crash never leaves a half-written result. All internal text is UTF-8, and every open() call in the codebase passes encoding= explicitly, enforced by a lint rule.

Test. Fixtures include byte-exact sample files: UTF-8 with and without BOM, cp1252 with accented names, UTF-16 exports from a mainframe, a file with Windows line endings, and one with an invalid byte in the middle. Tests use pytest’s tmp_path fixture for writes and assert the exact bytes produced, not just the decoded string, because round-trip bugs hide in the encoder. The suite runs with -X warn_default_encoding promoted to an error so an implicit default encoding fails CI, and a Windows CI runner catches locale-dependent assumptions that Linux developers never see.

Troubleshoot. UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe9 is the classic: a Latin-1 file opened as UTF-8; the fix is the correct encoding, never errors="ignore" in the import path, since that silently destroys customer names. A CSV with blank lines between rows on Windows is the missing newline="". “Too many open files” (OSError: [Errno 24]) is a file opened without with in a loop; -X dev turns the unclosed files into ResourceWarning with the allocation site. Partial output after a crash means buffered writes; os.fsync before the rename makes them durable.

The important consideration is that bytes have no encoding; something must declare it, and guessing is a last resort that must be logged.

47

Explain pytest and mocking in the context of Python.

pytest is the de facto testing framework for Python. Tests are plain functions named test_* that use the assert statement; pytest rewrites those asserts so a failure prints the values involved. Its distinctive features are fixtures, which provide reusable setup and teardown through dependency injection, and parametrisation, which runs one test body over many inputs. Plugins such as pytest-cov, pytest-asyncio and pytest-xdist extend it.

Mocking replaces a real dependency with a controllable stand-in so a test can run without a network, database or clock, and can assert how the dependency was called. The standard tool is unittest.mock: Mock and MagicMock record calls, patch swaps an attribute for the duration of a test, and autospec=True makes the mock reject calls that the real object would not accept. pytest’s monkeypatch fixture does the same swap with automatic undo, and pytest-mock exposes patch as the mocker fixture.

#--- payments.py ---
import requests
def capture(order_id, amount):
    r = requests.post(GATEWAY_URL, json={"id": order_id, "amount": amount})
    r.raise_for_status()
    return r.json()["reference"]

#--- test_payments.py ---
import pytest
from unittest.mock import patch
import payments

@pytest.mark.parametrize("amount", [1, 999_99])
def test_capture_posts_amount(amount):
    with patch("payments.requests.post", autospec=True) as post:
        post.return_value.json.return_value = {"reference": "ref-1"}
        assert payments.capture("o-1", amount) == "ref-1"
        post.assert_called_once()
        assert post.call_args.kwargs["json"]["amount"] == amount

The patch target is where the name is looked up (payments.requests.post), not where it is defined, because patch replaces an attribute on a module object. Mocks are best kept at the system boundary: HTTP clients, clocks, message brokers; mocking your own classes couples tests to implementation and makes refactors painful.

48

In a real project using Python, how would you apply, test, or troubleshoot pytest and mocking?

Apply. The test suite of an order service is layered. Fast unit tests mock only the boundaries, through a fake_gateway fixture built with Mock(spec=PaymentGateway) and a frozen_clock fixture that patches the time source, so domain logic runs in milliseconds. Integration tests use a real PostgreSQL started by a session-scoped fixture (testcontainers), with each test inside a transaction that is rolled back. conftest.py holds the shared fixtures, markers (@pytest.mark.slow) separate the tiers, and pytest -n auto from pytest-xdist runs them in parallel in CI. HTTP calls are stubbed with responses or respx rather than hand-built mocks so request matching and serialisation are realistic.

Test. Tests of the tests matter too: pytest-cov reports branch coverage with a floor the build must meet, and mutation testing (mutmut) on the pricing module occasionally confirms that assertions actually bite. For mock-heavy tests I use autospec=True so that a renamed method on the real class fails the test instead of letting a stale mock pass. Fixture scoping is reviewed deliberately: a module-scoped fixture that one test mutates is a flaky-test factory.

Troubleshoot. A mock that “does not take effect” is patched at the wrong place, typically patch("requests.post") when the module under test did from requests import post; patching mymodule.post fixes it. AttributeError: Mock object has no attribute with autospec means the production code is calling something the real object lacks, which is a real bug found. Tests that pass alone and fail in the suite share state through a module-scoped fixture or a module-level cache; pytest -p no:randomly versus pytest-randomly ordering, and --lf / -x to bisect, locate it. Flaky asyncio tests usually mix event loops across fixtures; a single asyncio_mode = "auto" setting and function-scoped loops stabilise them. pytest --durations=10 finds the slow fixtures that make developers skip running tests locally.

The important consideration is that every mock encodes an assumption about the real dependency; a contract test against the real service catches the day that assumption stops being true.

49

Explain profiling and performance in the context of Python.

Profiling means measuring where a program actually spends time or memory before changing it, because intuition about Python hot spots is usually wrong. The standard toolkit has three layers, and the workflow below moves from measuring to fixing.

Diagram: Measure timeit / benchmark, Profile cProfile, py-spy, Memory tracemalloc, Hot spot identified, Fix algorithm / C / cache, Re-measure
  • timeit and time.perf_counter() measure a snippet or a block; pytest-benchmark turns that into repeatable tests.
  • cProfile is a deterministic function-level profiler built in: python -m cProfile -o out.prof app.py, then pstats or snakeviz to sort by cumulative time. It adds overhead and attributes time to functions, not lines. line_profiler gives per-line detail for a chosen function.
  • py-spy is a sampling profiler that attaches to a running process without code changes and produces flame graphs, which makes it the tool for production. Python 3.12 added -X perf so Linux perf can see Python function names, and scalene combines CPU, memory and native-versus-Python attribution.
  • tracemalloc tracks allocations by source line and compares snapshots; memray adds native allocations and flame graphs.

Typical fixes, in order of pay-off: a better algorithm or data structure (O(n²) list membership to a set), moving work out of a loop, caching with functools.cache, batching I/O and database queries, pushing numeric work into NumPy or another C-backed library, and only then concurrency or a faster interpreter. Upgrading to 3.11+ alone typically gives 10–60% because of the specialising interpreter, and 3.13’s experimental JIT is opt-in and not yet a general speed-up.

50

In a real project using Python, how would you apply, test, or troubleshoot profiling and performance?

Apply. Performance work on a reporting API starts with observability, not profiling: request latency histograms and database query timings per endpoint show which 5% of requests cost 60% of the time. For a suspect endpoint I reproduce it locally with realistic data volume and run cProfile through snakeviz, or attach py-spy record --pid to a staging worker under load to get a flame graph without redeploying. Fixes are made one at a time, with a benchmark before and after. Typical wins: an N+1 query replaced with a join, a per-row datetime.strptime replaced with one parse per distinct value, json.loads on a 20 MB payload moved to orjson, and a dict keyed lookup replacing a nested loop.

Test. Performance regressions are caught by pytest-benchmark tests on the pure-Python hot paths, comparing against a stored baseline with a tolerance so noise does not fail the build, and by a nightly load test (locust or k6) against staging with p95 latency and error-rate thresholds.

Troubleshoot. A latency jump after a deploy is bisected by diffing the dependency lock as well as the code; a minor upgrade once swapped a C accelerator for a pure-Python fallback, visible in py-spy as time in the library’s Python frames. CPU at 100% on one core while others idle is the GIL with a thread pool on CPU-bound work. High latency with low CPU points at blocking I/O or lock contention, and py-spy dump shows the waiting stacks. Growing memory is approached with tracemalloc diffs; a flat profile with no hot spot usually means data volume, so the fix is pagination or a different query. cProfile inflates the cost of many tiny calls, so a sampling profiler confirms the ranking before anyone rewrites code.

The important consideration is that optimising without a measurement is guesswork; a profile, a baseline and a re-measurement are the minimum evidence for any performance change.

Answers tailored to your resume and job description

Add your resume and the job description in CrackItX's Settings. When the interviewer asks a Python question, Claude suggests an answer using your real projects and the role's requirements, in real time.

See pricing →