Concurrency & Async Programming in Python

Concurrency & Async Programming in Python – asyncio, Multithreading & Multiprocessing

If you’ve ever written a Python program that feels slow while waiting for an API response, downloading files, or processing heavy data you’ve already faced the need for concurrency.

Modern applications can’t afford to “wait.” They need to handle multiple tasks at the same time. That’s where concurrency and asynchronous programming come into play.

In this blog, we’ll understand in simple terms how Python handles concurrency using:

  • asyncio

  • multithreading

  • multiprocessing

And most importantly when to use each.

What is Concurrency?

Concurrency means handling multiple tasks during overlapping time periods. It doesn’t always mean tasks run at the exact same time but they make progress together.

Think of it like cooking:

  • While rice is boiling, you chop vegetables.

  • While vegetables cook, you prepare salad.

You’re not doing everything at the exact same second but you’re efficiently managing time.

1️⃣ Async Programming with asyncio

asyncio is designed for I/O-bound tasks — operations that spend time waiting:

  • API calls

  • Database queries

  • File reading/writing

  • Network requests

Instead of blocking the program while waiting, asyncio allows other tasks to run.

Example: Basic asyncio Program

import asyncio

async def task(name):
    print(f"Starting {name}")
    await asyncio.sleep(2)
    print(f"Finished {name}")

async def main():
    await asyncio.gather(
        task("Task 1"),
        task("Task 2")
    )

asyncio.run(main())

Both tasks run concurrently. Total time? Around 2 seconds — not 4.

When to Use asyncio

  • Building high-performance APIs (like FastAPI)

  • Handling thousands of concurrent connections

  • Web scraping

  • Real-time applications

asyncio is powerful, but it requires understanding async and await clearly.

2️⃣ Multithreading in Python

Multithreading allows multiple threads within the same process.

However, Python has something called the Global Interpreter Lock (GIL), which prevents true parallel execution of CPU-bound threads.

So what does that mean?

Multithreading is useful mainly for:

  • I/O-bound tasks

  • Background tasks

  • GUI applications

Example: Multithreading

import threading
import time

def task():
    print("Starting task")
    time.sleep(2)
    print("Task finished")

t1 = threading.Thread(target=task)
t2 = threading.Thread(target=task)

t1.start()
t2.start()

t1.join()
t2.join()

Threads are lightweight and share memory which makes communication easy but also requires careful handling of shared data.

3️⃣ Multiprocessing in Python

If you want true parallelism, especially for CPU-heavy tasks, multiprocessing is the answer.

Each process:

  • Has its own memory

  • Has its own Python interpreter

  • Bypasses the GIL

Example: Multiprocessing

from multiprocessing import Process
import time

def task():
    print("Processing...")
    time.sleep(2)

p1 = Process(target=task)
p2 = Process(target=task)

p1.start()
p2.start()

p1.join()
p2.join()

This is ideal for:

  • Data processing

  • Image processing

  • Machine learning tasks

  • Heavy computations

I/O-bound vs CPU-bound – The Key Difference

Understanding this changes everything.

I/O-bound Tasks

Spend time waiting.
Examples:

  • API calls

  • Database queries

  • File downloads

Best options:

  • asyncio

  • multithreading

CPU-bound Tasks

Spend time computing.
Examples:

  • Mathematical calculations

  • Video encoding

  • Machine learning training

Best option:

  • multiprocessing

Real-World Example

Imagine building a backend system:

  • Handling 10,000 API requests → Use asyncio

  • Sending emails in background → Use threading

  • Processing uploaded images → Use multiprocessing

Each tool solves a different problem.

Common Mistakes Beginners Make

  1. Using threads for CPU-heavy tasks

  2. Mixing async and blocking code incorrectly

  3. Ignoring synchronization (locks, queues)

  4. Overcomplicating simple programs

Concurrency adds power but also complexity.

Quick Comparison

Feature            asyncio                            Multithreading                    Multiprocessing
Best For           I/O-bound                            I/O-bound                      CPU-bound
Memory           Shared                            Shared                      Separate
True Parallelism           No                            No (due to GIL)                      Yes
Complexity           Medium                            Easy                      Medium

Concurrency is not about making code “fancy.” It’s about making it efficient.As a Python developer, mastering these three concepts gives you a serious advantage especially in backend development, data engineering, and system design.

Start small:

  • Write a simple async API

  • Create a threaded downloader

  • Build a multiprocessing data processor

Once you understand when to use each approach, you’ll write faster, more scalable Python applications.

Comments

Post a Comment

Popular posts from this blog

Middleware & CORS in FastAPI

  Introduction As your API grows, you may need to: Log requests Modify responses Handle cross-origin requests This is where Middleware and CORS (Cross-Origin Resource Sharing) come into play.They help control how requests and responses flow through your application. What is Middleware? Middleware is a function that runs: Before a request reaches your route After a response is returned Think of it as a pipeline layer between client and server. Request Flow with Middleware Client → Middleware → Route → Middleware → Response Creating Custom Middleware from fastapi import FastAPI , Request import time app = FastAPI () @ app . middleware ( "http" ) async def log_requests ( request : Request , call_next ): start_time = time . time() response = await call_next ( request ) process_time = time . time() - start_time print ( f"Request: { request . url} | Time: { process_time } " ) return response What This Middle...
Read more

Database Integration in FastAPI (SQLAlchemy CRUD)

  Introduction So far, we’ve built APIs without storing data.But real-world applications need a database to: Store user data. Manage products. Persist application state. In this blog, we’ll connect FastAPI with a database using SQLAlchemy and perform CRUD operations . What is SQLAlchemy? SQLAlchemy is a powerful Python ORM (Object Relational Mapper) that allows you to: Interact with databases using Python code. Avoid writing raw SQL queries. Work with models instead of tables. Step 1: Install Dependencies pip install sqlalchemy (For SQLite, no extra driver needed) Project Structure Update app/ ├── main.py ├── database/ │ ├── connection.py ├── models/ │ ├── item.py ├── schemas/ │ ├── item.py ├── routes/ │ ├── item.py Step 2: Database Connection app/database/connection.py from sqlalchemy import create_engine from sqlalchemy . orm import sessionmaker , declarative_base DATABASE_URL = "sqlite:///./test.db" engine = create_engine ( DAT...
Read more

Python Data Handling

Python Data Handling for Beginners In Python, data handling means storing, changing, and using data in different ways. In this post, you will learn: Lists Tuples Dictionaries Sets String Methods Type Conversion 1. Python Lists :Lists store many values in one variable.                fruits = ["apple", "banana", "mango"]                                                                       Method                          Use                                            append()          ...
Read more