Concurrency and Parallelism in Python 🚀🐍

Python provides multiple ways to handle concurrent execution of tasks, improving efficiency and performance for CPU-bound and I/O-bound operations.

1️⃣ Understanding Concurrency vs. Parallelism

🔹 Concurrency: Multiple tasks start, run, and complete in overlapping time periods (not necessarily simultaneously).
🔹 Parallelism: Multiple tasks run at the same time on multiple CPU cores.

Approach Type Best for

threading Concurrency I/O-bound tasks (e.g., network requests, file I/O)

multiprocessing Parallelism CPU-bound tasks (e.g., data processing, computations)

asyncio Asynchronous High-performance I/O (e.g., web scraping, database calls)

2️⃣ Multithreading with threading Module (Concurrency)

Python threads share the same memory space but run independently. However, due to the Global Interpreter Lock (GIL),
threads cannot execute Python bytecode in parallel, making it best suited for I/O-bound tasks.

✅ Example: Running Two Threads Simultaneously


import threading
import time

def print_numbers():
for i in range(5):
print(f"Number {i}")
time.sleep(1) # Simulating a delay

thread1 = threading.Thread(target=print_numbers)
thread2 = threading.Thread(target=print_numbers)

thread1.start()
thread2.start()

thread1.join() # Wait for thread1 to complete
thread2.join() # Wait for thread2 to complete
print("Threads finished")
✔ Best for tasks like web scraping, file downloads, or API calls.
✔ Threads share memory, reducing overhead but requiring thread safety (e.g., using locks).

3️⃣ Multiprocessing with multiprocessing Module (Parallelism)

The multiprocessing module creates separate processes, allowing true parallel execution without GIL restrictions. This is
ideal for CPU-bound tasks.

✅ Example: Running Multiple Processes in Parallel


import multiprocessing
import time

def square(n):
time.sleep(1) # Simulate computation
print(f"Square of {n}: {n * n}")

if __name__ == "__main__":
numbers = [1, 2, 3, 4, 5]
processes = []

for num in numbers:
p = multiprocessing.Process(target=square, args=(num,))
processes.append(p)
p.start()

for p in processes:
p.join() # Wait for all processes to finish

print("All processes completed")
✔ Uses multiple CPU cores for real parallel execution
✔ Best for heavy computations like image processing, machine learning, and large data analysis

4️⃣ Asynchronous Programming with asyncio (I/O-bound, Highly Scalable)
Unlike threads and processes, asyncio does not use multiple threads or processes. Instead, it switches between tasks when one is waiting, making it ideal for network operations, web scraping, or database queries.

✅ Example: Running Multiple Async Tasks Simultaneously


import asyncio

async def fetch_data(n):
print(f"Fetching data {n}...")
await asyncio.sleep(2) # Simulates a network request
print(f"Data {n} fetched")

async def main():
tasks = [fetch_data(i) for i in range(3)]
await asyncio.gather(*tasks) # Run tasks concurrently

asyncio.run(main())
✔ Highly efficient for handling thousands of network requests
✔ Best suited for async I/O tasks like fetching web pages, sending emails, or handling multiple client requests

5️⃣ When to Use What?

Method Best For Example Use Cases

threading Concurrency (I/O-bound) Web scraping, downloading multiple files, API requests

multiprocessing Parallelism (CPU-bound) Image processing, mathematical computations, ML training

asyncio Asynchronous I/O Handling thousands of network calls efficiently

6️⃣ Summary

🔹 Use threads for I/O-bound tasks that involve waiting (e.g., network requests).

🔹 Use multiprocessing for CPU-bound tasks that require intensive computation.

🔹 Use asyncio for high-performance I/O when handling thousands of requests.

    Date: 2025-03-24 00:00:00.000000