Support zsh

[eigengrau]

So, I’ve noticed quite a while ago that zsh support had been removed. So, since christmas is a time for miracles, I thought I might check back on any recent developments. From searching the issue tracker I take it that nothing has changed in this regard. So I figured I might open this issue so that patient, interested parties can subscribe to it and get notified in case someone feels like stepping up.

That is of course if you do deem zsh support generally useful. Otherwise please feel free to close the issue.

Thanks for making shellcheck!

[HaleTom]

shellcheck is amazing, and it is the reason I still write most of my scripts in bash.

I'd love it if zsh support could be added though!

[molovo]

As someone with quite a few big ZSH projects out in the wild, I'd definitely be interested in seeing this happen. I'd be very happy to contribute myself if someone could let me know what would be required to get it implemented.

[bittorf]

@molovo: first you have to ask yourself if the features of 'zsh' are a subset of other shells or if zsh has own features. look into the source for 'first citizen shells' and the handling of 'dash' and 'bash' and you will get a feeling, what is needed...

[koalaman]

This would require:

1. Implementing all the syntax that is unique to zsh, to allow scripts to be successfully parsed (Parser.hs). This requires some care to avoid strange parse errors for other shells.
2. Extend the variable flow tracing code and similar to support these new structures, so that shellcheck can tell when variables are modified and such (AnalyzerLib.hs).
3. Fixing or extending any tests that are not correct for zsh (Analytics.hs, Checks/*.hs).
4. Adding new and interesting Zsh specific checks.

Ksh shares basically all its syntax with Bash, which is why it's still supported even though almost no one uses it. Zsh required a lot more work for a similarly small number of users, which is why I unfortunately wasn't able to keep it up.

[LukeShu]

If zsh is ever supported, it should recognize #compdef and #autoload as shebangs for zsh, as these are magic headers used in zsh autocompletion helpers.

[kgizdov]

@koalaman I understand that bash is way more popular that zsh, but it still has a huge community. Would it be appropriate if you make a project/branch for zsh contributions so that users can contribute. Maybe put a note in the README and helpwanted flag here. If you set up the infrastructure, I'm sure people will bunch up to make this happen. I am willing to spare some time on this.

[brother]

@kgizdov on the other hand a PR with the start is pretty much the same. I assume that if you gather some people to do the work either they are welcome to the project or they will fork it.
The code won't be written and maintain by pure magic, as far as I can understand @koalaman will not take that responsibility himself.

[paulmelnikow]

I looked into this briefly a couple weeks ago. After I read through most of Parser.hs it was apparent that koalaman's step 1 would require deep knowledge of zsh syntax and bash syntax and a decent understanding of Haskell.

That made me wonder if zsh's parser (it's in C) could be used directly or modified slightly to emit an AST that could be interfaced to some high-level language constructs in Haskell or some other language. The wrapping is a bit involved, but is less work than reimplementing the parser.

I'm curious, has anyone on this thread worked with the zsh source?

[molovo]

@paulmelnikow I haven't worked with the source, but I have considered building an AST or something similiar (not for this purpose, but to support a coverage driver for zunit). I wonder if something could be built to tackle both issues at once?

[paulmelnikow]

@molovo Interesting idea!

[snoblenet]

Maybe the wiki could recommend a shellcheck alternative for zsh users? Because all Google searches based on variants of lint zsh take you back to shellcheck.

[paulmelnikow]

Unfortunately, there is no such equivalent. If anyone could make it happen, that'd be @molovo.

You may want to take a look at fish shell, which is better documented, more user-friendly, and developed on github.

[ghost]

I would like a support for Zsh but well Bash is still pre-installed in all major distro.

I code in Bash using Zsh...haha

[darthdeus]

I guess one could still use shellcheck for bash-ish zsh files? To be more specific, I have quite a few config files related to my ~/.zshrc and that's about the only place where I use zsh.

But being completely honest, I'm way more familiar with bash than with zsh, so sometimes when shellcheck is showing a warning, I'm not 100% sure if it applies to zsh as well.

[psprint]

I see Zsh thing happening if:

1. Shellcheck will be forked to e.g. ZShellcheck.
2. The authors do some effort to clean up the source to allow a fresh start for Zshell extensions, e.g. authors would disable support for ${!VAR} bash indirect-reference syntax and other things like this. It might be that some Bash property/feature required a hack or a bloated code somewhere, and this could be removed now.
3. A good code documentation will be available, and a channel to receive help from the authors.
4. Some people will get around the forked project, start small by e.g. implementing support for autoload functions (a feature of Zsh).
5. This way original shellcheck will sleep well, being undisturbed with Zsh-universe related actions.

I write this because I'm 3-years active Zsh (upstream) and Zsh-script-projects developer, and the above scenario covers my actual expectations to have a well organized environment/setup for working on a project.

BTW., Zsh can parse its code itself with a (z) substitution flag, I have based on it a Doxygen-like tool that works also with Bash scripts, check out significant-large Bash project (not mine) that uses it for Bash backend-scripts (the link points to a code-documentation generated with Zshelldoc). A code-documentation tool is somewhere in direction of linter tool.

[HaleTom]

There is so much commonality between {ba,z}sh that there would need to be a lot of importing upstream commits into the new project... @psprint What do you see as the advantages of a forked project?

Would it make sense to:

1. Move all common {ba,z}sh syntax to a meta-syntax, for example named: bazsh
2. Implement only the bash-specific extensions (eg things like ${var@Q}) in bash
3. Start the zsh-only syntax with a clean slate (importing only the common bazsh syntax)?

(I'm assuming one syntax can import/inherit from another syntax here - please correct me if I've misunderstood)

[psprint]

I think that without fork programmers will have to be real experts in Bash and Zsh. I learn Zsh for 3 years and sometimes there's still something that's new to me. If I imagine that I now would have to reach the same level in Bash but not through play & experiment like with Zsh, but with studying documentation, then I feel like running away from contributing.

My experience also tells me that structural programming doesn't match situation, where every action that a program has to make is also a result of condition like "Bash or Zsh". If I imagine codebase that's doing constant "Bash or Zsh" checking, I see something really severe, a mess. This mess can be controlled but this doesn't match Open Source programmers, IMO. Even well paid corpo programmers could fail and consciously doom project's codebase if someone would require of them that level of self control and care for details. Nobody would be able to prove them that they're making a crime for their comfort.

Forked project can just accumulate perfection in following Zsh syntax, it's a much simpler situation and it matches Open Source way of doing things.

[HaleTom]

I believe that the answer may depend on whether whether one shellcheck syntax-checker can inherit easily from another.

If it's possible, then perhaps something like this is possible:

1. Move all current bash syntax checkers into bazsh checkers
2. Create empty zsh syntax checkers inheriting from bazsh. These can be added to incrementally.
3. Allow bug reports to discover bash-only checkers in bazsh (and therefore in zsh), and move them to bash (which also inherits from bazsh.

This would hopefully avoid much of the shell version checking you describe.

I wonder how it is currently done for ksh vs bash that @koalaman mentions here?

@koalaman could you comment?

[danielbayley]

Wouldn't the simplest approach be to create a zsh branch or in the initial logic choose which parser to use based on the bang line #! /bin/zsh/#! /bin/bash etc…?

[psprint]

@HaleTom Ok the idea seems reasonable, it opens a way to do programming not whole-picture-grasping, which is pretty much the thing that I opt for. I think at this point, what's needed, is the Spellcheck authors to tell us how much the code is done in direct-structural way vs. indirect-with-high-degree-of-freedom way. I mean, you wrote about kind of syntax-"backends" that can be moved level-up or level-down between general-shell and specific-{bash,zsh} levels, to gradually separate things, but in reality I suspect this will have to be coded first, and I wonder how much parsing-knowledge or parsing-certainty/comfort would it require to actually approach this task.

[koalaman]

I appreciate your assumptions that ShellCheck is an enterprise level project with smart abstractions and a clever design, but you're really overestimating both it and me. Apart from being written in Haskell, ShellCheck is very simple and straight forward.

It essentially just has one parser and one set of checks that are applied to the result.

The parser handles all syntax for all shells the same way. If you use a Bash specific feature in Dash, such as for ((;;)), then it'll be parsed and interpreted as that Bash feature, and will get all corresponding Bash warnings. There's hopefully also a check that warns that it's unsupported in Dash, but that doesn't change how ShellCheck processes it.

There's no reason why you couldn't additionally recognize Zsh for x (1 2 3) and do the same. Note especially that it's not necessary to implement more than a single statement type at a time before you can run it and see the effects.

As for the checks, they don't have groups or stages. The only structure imposed is optional, and is purely for convenience and efficiency (like registering to handle a command name instead of having to iterate the AST to look for them). If a check is specific to a different shell, it'll just check and do an early return.

I doubt any of that would be a concern though. 95%+ of checks that apply to Bash also apply to Zsh (fgrep isn't less deprecated for example). There's probably <5 that don't, and those could just return early as well.

[psprint]

I see, from that point of view it looks easy. With the possible problems anticipated by me, I was thinking about things that are more subtle than an additional syntax. Like for example word-splitting being the default in bash, Zsh whole array referencing with $array instead of ${array[@]} (however: it works like in bash if ksharrays option is set), array indexing starting at 0 not 1, etc. I have had an impression that there is much such things, but I cannot be sure about this, would have to sit down and think about this a little longer.

[psprint]

@koalaman; Can it be said, that after implementing checks for a most easily-occurring mistakes, Shellcheck took a path of implementing checks for a more complex & rare, more specific, narrow, fine-grained mistakes? And that that's why the execution of a shell script has been implemented – to be able to dig into the states of the code and catch the fine-grained mistakes?

[phdoerfler]

It's a poor metric in general. All it shows is that bash raises more questions than zsh (on Stack Overflow). If anything you could use that to make the point that zsh is less obscure than bash but for the reasons @vitorgalvao outlined even that makes not much sense.
Speaking of bad scripting languages: I've seen a similar metric used to underline how much more popular Python was compared to R and the only thing that does is to show how much more questions Python raises over R :P

[LwsBtlr]

zsh and bash are close enough that asking about something in bash will more likely get answers than asking about the same thing in zsh, so the zsh questions tend to be zsh specific issue that differentiate it from bash.

[bl-ue]

Probably the biggest reason that this hasn't been implemented yet is that the project is written in Haskell, Lot's of folks don't know it since it doesn't follow in the train of standard C-based programming languages sigh 😞

[TinCanTech]

So write one in in C and stop complaining.

[jhunterkohler]

@TinCanTech The functional squad did not like that one.

[TinCanTech]

@HunterKohler .. is that like Hunter-Koala or Hunter-killer ?

I think @koalaman has done more then enough already. See his reply above ..

I think asking one man to port this entire project to a new language C, to satisfy a dozen-or-so people, who did not have an alternative to my suggestion (at least they have not mentioned it so far .. ) is a little steep.

Perhaps Haskell is the correct language for this project ; perhaps more people should try to learn it and contribute ..

Please, down-vote me again, if you do not understand what it takes to create shellcheck in the first place.
Otherwise, Please, down-vote me again and put a reason ..

[vitorgalvao]

Please, down-vote me again, if you do not understand what it takes to create shellcheck in the first place.
Otherwise, Please, down-vote me again and put a reason ..

You got them the first time (I bet) because your reply was unhelpful and rude without provocation.

I think asking one man to port this entire project to a new language C, to satisfy a dozen-or-so people, who did not have an alternative to my suggestion (at least they have not mentioned it so far .. ) is a little steep.

If that’s what you took from @bl-ue’s comment, you’re extrapolating. They lamented that so few people know the language, they did not ask for a rewrite.

Downvotes are meaningless on GitHub. Their only effect is increasing a number near an icon. Let’s move on and keep the conversation focused on adding (or not) ZSH support to Shellcheck.

[TinCanTech]

My apologies for any offence that I may have caused.

One might equally lament any number of things (eg: JFK) .. but it serves no purpose.

Let’s move on and keep the conversation focused on adding (or not) ZSH support to Shellcheck

Agreed.

[nacho4d]

Hi, We are in 2022, a bit more than 5 years since this issue was originally created.

I would like to see zsh support too. From all this thread I get is it not easy task but still wanted. zsh is the default in macOS. (The only reason I don't heavily use zsh in my scripts is because shellcheck does not support it. But I want to use zsh!)

[vitorgalvao]

We are in 2022, a bit more than 5 years since this issue was originally created.

That is irrelevant. @koalaman owes us nothing and it’s their right to never work on this. That this issue is still open is a courtesy we’ve been given.

This is a free project open for all to see. If you really want zsh support, the options are:

• Do it yourself and submit a pull request.
• Ask or pay someone else to do it.
• Offer to pay @koalaman for their time, as an incentive to get it done.

Anything different is at this point screaming into the void. If you want it to happen, make it happen.

[andrewferrier]

FWIW, I have started using zsh -n $FILENAME as a syntax checker integration inside my editor. It does not do anything as close as to what shellcheck does, but it will output the first error it finds in a pattern similar to shellcheck. So it's better than nothing.

(In case anyone is interested, it's a custom diagnostic source for null-ls in NeoVim - happy to share more details if anyone's interested).

Thanks to @koalaman for all his hard work on bash for shellcheck. It's saved many wasted hours debugging problems with shell scripts.

[mcandre]

Hi Andrew, Yes, I plan for my "stank" shell script linter to do the same, running the -n option based on a file's interpreter (or known dotfile association for standard config scripts).

…

On Thu, Feb 3, 2022, 7:24 AM Andrew Ferrier ***@***.***> wrote: FWIW, I have started using zsh -n $FILENAME as a syntax checker integration inside my editor. It does not do anything as close as to what shellcheck does, but it will output the first error it finds in a pattern similar to shellcheck. So it's better than nothing. (In case anyone is interested, it's a custom diagnostic source for null-ls in NeoVim - happy to share more details if anyone's interested). Thanks to @koalaman <https://github.com/koalaman> for all his hard work on bash for shellcheck. It's saved many wasted hours debugging problems with shell scripts. — Reply to this email directly, view it on GitHub <#809 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAABJRBQR4SVU342QF6JPPLUZJ6Y5ANCNFSM4C2VX25Q> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>. You are receiving this because you were mentioned.Message ID: ***@***.***>

[pboushy]

@koalaman I understand that bash is way more popular that zsh, but it still has a huge community. Would it be appropriate if you make a project/branch for zsh contributions so that users can contribute. Maybe put a note in the README and helpwanted flag here. If you set up the infrastructure, I'm sure people will bunch up to make this happen. I am willing to spare some time on this.

@koalaman - I'm willing to put some significant time into this, but I definitely need some help on where to start in this to really contribute. If you're open to it, I think the recommendation here would be a good starting point.

[koalaman]

@pboushy I appreciate your enthusiasm, but to set expectations, this is an abnormally large and all-encompassing contribution that dwarfs all previous ones. I would expect it to take months of full time work. As you can tell from the speed of my replies, I'm don't have nearly the time I wish I did to help out either. If you'd still like to give it a whack, check out the DevGuide, and have a look at the quite readable Debug.hs that makes it easier to develop/test interactively.

A good first step might be to try to implement the short forms of if or for

[RexYuan]

Should the tool suggest that, when it detects zsh shebang, there is an ongoing conversation on this topic?

[psprint]

I've created a new linter for zsh called 🧽 zsh-sweep. It's at http://GitHub.com/psprint/zsh-sweep