Skip to content
This repository has been archived by the owner on Apr 16, 2021. It is now read-only.

Narann/threadlist

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

10 Commits
 
 
 
 
 
 

Repository files navigation

threadlist

A thread loop "limiter" module for Python.

More details using:

import threadlist
help(threadlist)

ThreadList

An extended Python list to handle (and limit) thread executions.

When you run many slow threading.Thread() instances, you can reach the maximum number of thread your system can support. This special list provide some controls on how threads are started.

The way threads are handles by this list is time based. This is efficient for slow threads (slower than 0.1 sec) but can be less efficient on multiple very fast threads.

import threading
from threadlist import ThreadList, TimeoutError

threads = ThreadList()
for _ in xrange(10):
    thread = threading.Thread()
    threads.append(thread)
threads.max_count = 4
threads.total_timeout = 15
try:
    threads.run() # wait until every thread is terminated
except TimeoutError:
    print "Threads Timeout reached!"

A typical big images conversion example:

import threading, multiprocessing
from threadlist import ThreadList

image_paths # a huge list of image file to convert

threads = ThreadList()
for image_path in image_paths:
    thread = SlowConversionThread(image_path)
    threads.append(thread)
threads.max_count = multiprocessing.cpu_count()
threads.start()
# continue your script here...
threads.join()

About

A thread loop "limiter" module for Python

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages