Reviewed by Aditya Kumar · Last reviewed 2026-08-08
To log the execution time of a function, you can create a Python decorator that wraps the target function, records the time before and after its execution, and then logs the duration. This pattern is…
This easy-level Python/Coding question appears frequently in data engineering interviews at companies like American Express. While less common, it tests deeper understanding that distinguishes strong candidates.
Start by clearly defining the core concept being asked about. Interviewers want to see that you understand the fundamentals before diving into implementation details. Structure your answer with a definition, then explain the practical application with a concise example. The expert answer includes a code example that demonstrates the implementation pattern.
To log the execution time of a function, you can create a Python decorator that wraps the target function, records the time before and after its execution, and then logs the duration. This pattern is invaluable for performance monitoring and debugging in data pipelines.
A decorator is a higher-order function that takes another function as an argument and extends or modifies its behavior without explicitly changing its source code. The core steps involve:
wrapper function that will execute the original function.time.perf_counter() for high-resolution, process-specific timing, ideal for measuring short durations accurately. Avoid time.time() which is system-wide and less precise for this use case.logging module instead of print for better control over log levels, destinations, and structured output.functools.wraps copies important metadata (like __name__, __doc__) from the original function to the wrapper, ensuring introspection and debugging tools work as expected.import time
import functools
import logging
# Configure basic logging for demonstration
logging.basicConfig(level=logging.INFO, format='%(asctime)s - %(levelname)s - %(message)s')
def log_execution_time(func):
@functools.wraps(func)
def wrapper(args, *kwargs):
start_time = time.perf_counter()
result = func(args, *kwargs)
end_time = time.perf_counter()
duration = end_time - start_time
logging.info(f"Function '{func.__name__}' executed in {duration:.4f} seconds.")
return result
return wrapper
This decorator is highly useful in data engineering for profiling ETL steps, identifying bottlenecks in complex transformations (e.g., a Spark UDF, a Pandas operation, or a dbt model hook), and monitoring the performance of batch jobs. By applying this to specific functions within a data pipeline, engineers can pinpoint slow components, similar to how one might analyze Spark job stages or Snowflake query profiles.
The primary trade-off is minimal overhead from the timing and logging operations. While negligible for most functions, for extremely high-frequency, low-latency calls, this overhead could become a consideration. For production, ensure logs are directed to a centralized logging system (e.g., Splunk, ELK stack) rather than just stdout.
Discuss the importance of functools.wraps for maintainability and debugging, and the choice of time.perf_counter for accuracy. Emphasize how this pattern aids in building observable and performant data pipelines.
Red Flag: Forgetting @wraps (breaks introspection). Pro-Move: 'We use this + logging to metrics—every ETL step reports duration to DataDog.'
Some links below are affiliate links. If you buy through them we may earn a small commission at no extra cost to you — it helps keep DataEngPrep free.
According to DataEngPrep.tech, this is one of the most frequently asked Python/Coding interview questions, reported at 1 company. DataEngPrep.tech maintains an editor-reviewed database of 1,863 data engineering interview questions across 7 categories.