-
Notifications
You must be signed in to change notification settings - Fork 8
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
temp file race conditions when invoking tile() in parallel #18
Comments
This request I agree it would be great for parallel processing. I don't have time to dig into this one myself very soon. But if you can make a PR, I can review it, at least to get the process started. Are you suggesting append a random string of characters to It sounds like (2) can be addressed independently from (1). Regards, |
Yes, sorry I just edited my OP. For 1) I suggest I have (1) implemented in my fork and I am testing with some of my projects. Will PR when I'm happy with it (though note my fork also implements #19). |
Thank you! That sounds good to me, feel free to combine it all in one PR. |
update: changes for number 1 seem fine -- it's number 2 that's biting me still. |
@achubaty Hi Alex, it's been a while; I know this was merged, but are we good to close the issue or is there still something outstanding? |
Hi Matt, the first part is completed, but I haven't come back to deal with the second part of this issue ('attempts to use the same temp file when rasters are not one of the supported filetypes') - so this piece is still outstanding. |
@leonawicz Hi again, I started working on this but encountered an issue when running tests using the latest |
with ropensci#18 requires fix for rspatial/raster#315
Hi, yeah I encountered a similar problem with a couple unit tests involving raster stacks last week and had to skip them for now. There may be an option to remove My next goal is to at least republish to CRAN the current version on master, to address critical dependency issues #22 If we can fold in completion of this issue with the next release, that would be a bonus, but it's also fine if we don't. |
CRAN submission is offline until August 7. I will submit afterward. But everything looks to be passing. :) |
@leonawicz this has been fixed upstream in |
Calling
tile()
in a parallel (i.e., multiple concurrenttile
calls) produces the following problems:g2ttmp
directory.In order to make this work, each nodes needs its own tmpdir. The following change does the trick, provided the rasters are all of the supported type:
Unfortunately this is a more involved fix because that file name is hardcoded in multiple places outside of
tile()
, so would need to pass the temp file name through every other function that checks for that file. Otherwise, I'd usetempfile("tmp_raster_", fileext = ".tif")
here too.The text was updated successfully, but these errors were encountered: