Ignore uneditable files in the file picker

[EpocSquadron]

This is probably incomplete, in that it only touches the file picker and not the suggestions in the open command, and it also only lists out common compressed file extensions. It should probably include a lot more things that are not editable, like iso, dmg, and more. I'm also unsure if this approach is preferred as file extension isn't 100% reliable, but I feel the blacklist approach is better than whitelist as we don't want to inadvertently prevent people from editing files that are perfectly editable.

This is probably my first contributed rust code ever. I'm unsure if I did things the best way or not, so style critique is appreciated as well.

[pickfire]

Thanks a lot for sending in the pull request. It does seemed to be good to have a simple version like this without configuration at first.

What is your use case? Currently ignore Walk already ignores stuff in git ignore. Are you editing stuff that is not in a git repository?

I wonder how grep_searcher::BinaryDetection interacts with ignore within ripgrep which lets it checks only text file (ripgrep --text). From what I see it seemed like content based more than file extension based.

[EpocSquadron]

@pickfire There are a number of cases I run into where this affects me:

• In a newer repository that hasn't added certain files to gitignore yet.
• In a repository that has some binaries tracked (for example web builds with static images, or zip files to be distributed).
  • Parcel 2, recently released, for example allows you to compile out brotli/gzip compressed bundles in case your server doesnt apply it transparently or its too expensive to compress on the fly.
• Occaisonal side-effect files from things I'm playing around with adding to a repo but am not ready to commit (and may never be), such as adding Phive to manage phar (php archives) for managing linter/formatters, static analyzers that shouldn't (for purposes of incompatible configuration between versions) be installed globally and cause dependency conflicts if attempting to use the standard composer dependency manager.
• Occasionally I uncompress zips of plugins or things like that for eg wordpress plugin installations (offline), and those sometimes contain blobs for various things.
• I have a bad habit of keeping around database snapshots in my web projects without adding *.sql.gz to the gitignore. I'm pretty handy with git so i just commit around them.

[EpocSquadron]

I took a look at the binary detection stuff in ripgrep and it looks like it is indeed inspecting the contents and testing if it is text or not. How should this get brought in? It looks like that subcrate isn't really quite intended to be used outside of ripgrep itself yet. I would love to see the picker restrict itself to text files though, that would be a better approach than this pull request for sure. I've also learned a lot more about proper result/error propogation rather than the verbose match arms I used here, but I'm not motivated to refactor if a better approach is available.

[EpocSquadron]

Looking further it looks like grep-searcher is already depended on in helix-term for global search. I'm not immediately certain how to pull in just the binary detection in the walk logic. It seems like it might be inefficient to build a searcher when we want all the files.

[archseer]

Sorry for the delay, it seems I forgot to review the PR previously. I think this is a good first pass to eliminate potential matches ahead of time, for binary files we'll just handle those in the picker before rendering a preview. (We also want to filter any files over 20MB probably)

[EpocSquadron]

If you think it's a good first pass then I'll refactor and rebase. Hopefully I can get to it this week.

[pickfire]

Suggested change

	// We want to exclude files that the editor can't handle yet
	let mut type_builder = TypesBuilder::new();
	let mut walk_builder = WalkBuilder::new(&root);
	let walk_builder = match type_builder.add(
	"compressed",
	"*.{zip,gz,bz2,zst,lzo,sz,tgz,tbz2,lz,lz4,lzma,lzo,z,Z,xz,7z,rar,cab}",
	) {
	Err(_) => &walk_builder,
	_ => {
	type_builder.negate("all");
	let excluded_types = type_builder.build().unwrap();
	walk_builder.types(excluded_types)
	}
	};
	// We want to exclude files that the editor can't handle yet
	let mut type_builder = TypesBuilder::new();
	let mut walk_builder = WalkBuilder::new(&root);
	let glob = "*.{zip,gz,bz2,zst,lzo,sz,tgz,tbz2,lz,lz4,lzma,lzo,z,Z,xz,7z,rar,cab}";
	let walk_builder = if type_builder.add("compressed", glob).is_ok() {
	type_builder.negate("all");
	let excluded_types = type_builder.build().unwrap();
	walk_builder.types(excluded_types)
	} else {
	&walk_builder
	};

Would this be better?

[kirawi]

Possibly related sharkdp/fd#749

[archseer]

We don't use fd, it's not available as a library.

[kirawi]

I meant the solutions they proposed.

[archseer]

We'll basically do this: https://docs.rs/grep-searcher/0.1.8/grep_searcher/struct.BinaryDetection.html

Pre-read a small buffer at the start of the file and scan for a 0x0 (null) byte. Looks like https://github.com/sharkdp/content_inspector/blob/master/src/lib.rs does a similar thing.