Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

0.4.0 questions #179

Open
pellepl opened this issue Oct 10, 2017 · 37 comments
Open

0.4.0 questions #179

pellepl opened this issue Oct 10, 2017 · 37 comments

Comments

@pellepl
Copy link
Owner

pellepl commented Oct 10, 2017

This issue is for asking on the upcoming 0.4.0 version.

0.4.0 is under construction. This is a full rewrite and will change the underlying structure. Hence, it will not be compatible with earlier versions of the filesystem. The API is the same, with minor modifications. Some config flags will be removed (as they are mandatory in 0.4.0) and some features might fall away until 0.4.1.
@igrr
Copy link
Contributor

igrr commented Oct 10, 2017 via email

@pellepl
Copy link
Owner Author

pellepl commented Oct 10, 2017

@igrr: Yes, the config options regarding file system structure are decided runtime. There is no need to typedef sizes for page indices and such in 0.4.0, this is determined compile time by spiffs itself (depending on file system size, block size, and page size). There will be compile time switches, but the goal is to keep these to functionality only, without affecting file system structure. The exceptions are file name size and optional meta data length - these must be compile time limited.

The images will be self-describing as a part of this also. The old SPIFFS_MAGIC* config option disappears in 0.4.0 and corresponding functionality is always on.

@benVolatiles
Copy link

Will there be an option/tool to convert existing SPIFFSs to the new version?

If not, is the old 0.3.x branch still receiving updates for devices that already have SPIFFS deployed on them?

@pellepl
Copy link
Owner Author

pellepl commented Oct 10, 2017 via email

@rojer
Copy link
Contributor

rojer commented Oct 16, 2017

If requested

ok, let me put in a request then :)
we would like to stay on latest SPIFFS, but we have older builds of user firmware to consider during OTA. luckily, our OTA does not require the migration to be performed in place, so being able to mount old FS read-only to copy the data over to the new FS would be sufficient. i'm even fine with shipping 0.3.x and 0.4.x code together to use the right one as appropriate, provided there is a way to determine which one should be used.

@sensslen
Copy link
Contributor

Does SPIFFS 0.4.x support asynchronous operation?

With asynchronous I mean that you can e.g. ask for some data to be written and get a callback when the data is actually written.

This could be done e.g. by requiring a asynchronous Flash Driver...

@pellepl
Copy link
Owner Author

pellepl commented Oct 16, 2017 via email

@pellepl
Copy link
Owner Author

pellepl commented Oct 19, 2017

@rojer

so being able to mount old FS read-only to copy the data over to the new FS would be sufficient.

That's good. Still, you mean on target right?

@rojer
Copy link
Contributor

rojer commented Oct 19, 2017 •
edited
Loading

yes, on target. OTA will come with new spiffs code and new fs image. new code boots and mounts the new fs, then it needs to mount the old FS (in r/o mode) and copy over some files.

@pellepl
Copy link
Owner Author

pellepl commented Oct 19, 2017

@rojer ok, got it. Do you have room for both old and new fs?

@rojer
Copy link
Contributor

rojer commented Oct 19, 2017

yes, we reserve 2x the space for code and fs, to make sure we can revert in case of failed OTA.

@uniederer
Copy link

Do you have anything in mind allowing for "background garbage collection" in a way that the erase (especially) as well as the preparation of erased pages is not done synchronously?

Using SPIFFS 0v3 we found out that GC takes place in the middle of some write process. As a result the write has excessive delays. Even calling the GC during the "idle phase" didn't help there for some weird reason. So we came to the conclusion it might be beneficial if one could kick GC early and let it run in background rather than waiting until we desperately need some memory. Of course this could introduce other issues like Flash is busy while one desperately wants to flush a page, but maybe it's worth some thinking about it.

@spe-ciellt
Copy link

This is maybe pushing it, but journaling?

@FrederikTB
Copy link

@uniederer You can call in the garbage collector early by using SPIFFS_gc. Does this not work?

@gabrielssanches
Copy link

Directory support?

@everslick
Copy link

I'm missing an API function to truncate a file to a specific size without having to create a copy first.

@uniederer
Copy link

@FrederikTB As i wrote in my original posting, calling the GC early (i.e. during the idle phase of the system) won't prevent SPIFFS from kicking the GC again upon the next write. The second Problem is the blocking nature of the GC run blocking everything in the System.

@PicoEmma
Copy link

Is there an approximate release date for 0.4.0?

@everslick
Copy link

Additionally I think a common use case is appending files (for writing logs or CSV files ...), so this should be as fast as possible. Currently I have big issues with that.

@temach
Copy link

temach commented Jan 19, 2018

Indeed some way to simplify logging would be great, such as functions to truncate a file and for fast apppending.

@rphedman
Copy link

rphedman commented Mar 1, 2018

Currently SPIFFS_check() time appears to be quadratic on filesystem size. This makes it pretty slow on larger filesystems.

I guess this is not likely to change? My current plan is to make multiple small filesystems to get around that.

Very much looking forward to 0.4.0!

@ShareCat
Copy link

ShareCat commented Apr 4, 2018

can V0.4.0 use less MACRO DEFINE? u can use functions instead of MACRO DEFINE, functions is more understandable, and save MCU flash.

@yogeshk19
Copy link

Is there an approximate timeline for when this 0.4.0 would be released?

@ShareCat
Copy link

ShareCat commented Nov 6, 2018

Looking forward to power failure safe mode.

@eyalasko eyalasko mentioned this issue Jan 22, 2019
Open
@GeorgeFlorian
Copy link

Is this still coming ?

@pellepl
Copy link
Owner Author

pellepl commented Feb 4, 2019

Sorry, I cannot say if nor when. Due to personal reasons all spiffs dev is on ice.

@GeorgeFlorian
Copy link

Sorry, I cannot say if nor when. Due to personal reasons all spiffs dev is on ice.

Don't you worry. We, plebs, will manage ! Thank you for your great work ! Get better !

@pellepl
Copy link
Owner Author

pellepl commented Feb 5, 2019 via email

@hzm476172557
Copy link

Hey dude
are you still working on it?
You have done a great job,and we hope you can carry on~
Greetings from China.

@marcwolf1960
Copy link

Hi.
Currently the only way to copy a file from a PC to the SPIFF is via the Python utility, or to enable a full web server. This can cause issues when you want to update a file on the SPIFF from a PC, where the only connection is serial.
Could there be a function or protocol where you could open a file on the SPIFF and stream from the serial directly in to the file and then close it.
Many thanks

@jago85
Copy link

jago85 commented Aug 9, 2019

Hi.
Currently the only way to copy a file from a PC to the SPIFF is via the Python utility, or to enable a full web server. This can cause issues when you want to update a file on the SPIFF from a PC, where the only connection is serial.
Could there be a function or protocol where you could open a file on the SPIFF and stream from the serial directly in to the file and then close it.
Many thanks

Hi marcwolf1960,
of course there could be such a protocol. This would be very application specific and does not necessarily have to be part of a library. I implemented a protocol over UART for my application and it would not be much work for you to do the same. My protocol was a quick shoot. It does not need to be very stable and handle every possible error. It works as follows:

  1. Host sends a 16 byte GUID (for synchronization)
  2. Device sends 0x00 as ack
  3. Host sends 1 byte length of the file name
  4. Device sends 0x00 as ack
  5. Host sends the filename
  6. Device opens or creates the file and sends 0x00 as ack
  7. Host sends 4 bytes file size
  8. Device sends 0x00 as ack
  9. Host sends chunks of 1024 bytes (or remaining data if less than 1024 bytes)
  10. Device writes the data to the file and sends 0x00 as ack (repeat previous step if more data is to be received)
  11. when all bytes are received the device closes the file and waits for the next GUID

@pellepl
Copy link
Owner Author

pellepl commented Sep 6, 2019

@hzm476172557 thanks for the kind words. No, not working on it now. My spare time is too sparse for this nowadays. Not saying I'll never return to it though.

@ali80
Copy link

ali80 commented Dec 6, 2019

You have done great service for the community, hope your are doing well and hear from you soon

@NicoPy
Copy link

NicoPy commented Sep 16, 2022

Hi,

Any news about V0.4.0 ?
I'm asking because I'd like to translate your excellent work to another language (Ada). The translation process is quite hard and long. If V0.4.0 is ready, I'll switch to it directly.

Regards,
Nicolas

@pellepl
Copy link
Owner Author

pellepl commented Sep 16, 2022 via email

@NicoPy
Copy link

NicoPy commented Sep 16, 2022

Hi,

OK. So, I'll go with the current version.
Thanks for your quick answer.

Regards,
Nicolas

@bmcdonnell-fb
Copy link

@pellepl,

Sorry but spiffs is dormant since years ago. Still have no plans of picking it up.

Perhaps you can update the README.md to reflect this? Particularly the "NOTICE" section, which says,

0.4.0 is under construction. ...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests