-
Notifications
You must be signed in to change notification settings - Fork 55
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Multithreaded Streams #4
Comments
Great! Jonah! Can't wait for the parallelised streams 👍 :) |
I think your job with this project is great! Parallelism are one obvious next big thing for a functional framework since one of the biggest selling points of functional programming is to give to developers a robust and cheap access to parallel processing. The reasons OOProgramming is loosing the 'holiness aura' it used to have in the past years are:
Here come the big U-turn of many developers about OO, including myself. Do you have ideas how to implement with simplicity multi threading? What about the following? // This sample takes an huge amount of words, and put them into a normalized, sorted and deduplicated vector:
myWords = load_from_somewhere();
auto my_normalize = [ ](const std::sting& word) -> std::string {
...
return normalizedWord;
};
auto maxWorkers = 4; // Takes up to 4 CPUs
auto bucketSize = 10000; // Every internal parallel task will have 10k elements.
auto myVucabulary = myWords | parallel(map_(my_normalize) | distinct(),
numWorkers, bucketSize)
| distinct()
| to_vector(); If you want I can help you with or we can work together and design the best solution. |
Allow parallelized streams (with a more limited set of operators).
The text was updated successfully, but these errors were encountered: