Display of diagnostic messages #150
Comments
|
Wow what a beautiful error message! If someone wants to work on beautifying the error message rendering that would be great. I will probably get to it eventually :-) |
c42f
added
error messages
|
Hello everyone, I'm the developer of Term.jl. I just wanted to say, this issue inspired me to add this kind of "annotations" in the image above to Term as a new feature (see: FedeClaudi/Term.jl#185). This is what it looks like: Annotation(
highlight("var::Int = 2x + \"a\""),
"2x + \"a\"",
"This seems like a bad idea";
style="bright_yellow"
) |> print
If it's something you'd find useful, let me know! Edit: @c42f perhaps there's a way to integrate Term's stacktraces to help beautify messages? I've spent a lot of time on the error message system in Term and, while not perfect, I think it might help. This is what an error message currently looks like |
|
@FedeClaudi this looks beautiful ❤️ (Sorry didn't mean to ignore you, I just missed your message.) If you'd like to help develop code in JuliaSyntax for rendering error messages that would be amazing! The key thing we need in this package is highlighting and annotating byte ranges within a file. You can see how this currently works here: JuliaSyntax.jl/src/diagnostics.jl Line 44 in 45d8431 Dependencies are a tricky thing for JuliaSyntax.jl as we need to vendor it into Base and eventually make it part of bootstrap somehow. So I wonder how we'd deal with that. Depending on Term.jl could be a problem if we have to vendor Term and all its dependencies into Base as well. |
Got it, integration with Keep up the good work, |
Maybe we could have hooks so that Term could be installed as an optional dependency? I'd like to make it easy to experiment with pretty printing. |
|
That could be a nice way to go about it! |
|
I'd be interested in contributing to making better diagnostics possible! Do we have a plan for how to tackle this? Btw, I thought this article on the rust blog was instructive: https://blog.rust-lang.org/2016/08/10/Shape-of-errors-to-come.html What are your thoughts on conducting a informal survey first to see what styles people would prefer and why? I seem to remember that some folks from the rust team did a survey after implementing diagnostics a number of different ways, and I seem to remember reading about it in a blog post or article but I can't seem to find that now. |
Great!
There's two aspects to this:
I like this article, I think I've read it before. The key point is that errors should focus on the code you wrote and this is something I've tried to do already in this package: errors are always presented with a chunk of source code highlighted for context. In contrast to the rust blog, JuliaSyntax doesn't mix the error message with the code - I'm trying preserve the visual shape of the code to visually orient the user in the minimum amount of time. So the error message is separate for now. A problem here is the limitations of ansi color highlighting
Unicode makes the ascii art options difficult to do in full generality. I like this blog post from the Fennel language discussing why ANSI highlighting is good when dealing with unicode text in diverse languages http://technomancy.us/198 By the way, I just discovered that some terminals like kitty support ANSI underline styles. Cool!
I think it would be best to spend some time researching what other well-regarded compilers are doing first and build up a set of options. There's been a lot of work put into this by other people and we should learn from that. Also if we can find that existing survey it would be very useful and much easier than doing our own. |
|
Thanks for the detailed comment! I'll respond when I get a chance to go through the links but just wanted to reply to one thing:
I was searching high and low for the article, and it turns out it was a YouTube video from just last month! I must have watched it and promptly forgot that it was a video. Link here: https://www.youtube.com/watch?v=Iv3UuGdB0TM It is not exactly a survey but the video is very informative, and I recommend watching it. The video is by Jonathan Turner, the same person that wrote the article from the comment. I'm wondering if it worth pinging |
Answering this question - yes! Here's a demo of one way to implement an "open in browser" functionality for error messages: Here's a webpage which we will target: <!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8" >
<meta name="viewport" content="initial-scale=1.0, maximum-scale=1.0, minimum-scale=1.0, user-scalable=no, width=device-width">
<title>Ansi Demo</title>
<link rel="stylesheet" href="https://muffinman.io/skyblue/css/skyblue.css">
<style>
body {
background: #fdfdfd;
padding-bottom: 100px;
}
.content {
position: relative;
margin: 0 auto;
padding: 20px;
max-width: 840px;
}
#source {
margin: 20px auto;
background: black;
}
</style>
<script src="https://cdn.jsdelivr.net/gh/drudru/ansi_up/ansi_up.js"></script>
</head>
<body>
<div class='content margin-y-40'>
<h1>Highlighted</h1>
<pre id='source'>
some code
or other
</pre>
<button onclick="copy_html_to_clipboard()">Copy html to clipboard</button>
<script>
const query = window.location.search
console.log(query)
const url_params = new URLSearchParams(query)
const ansi_str = url_params.get('ansi')
console.log(ansi_str)
const ansi_up = new AnsiUp
const src_div = document.getElementById("source")
const html = ansi_up.ansi_to_html(ansi_str)
console.log(html)
src_div.innerHTML = html
function copy_html_to_clipboard() {
navigator.clipboard.writeText(src_div.outerHTML)
}
</script>
</body>
</html>Here's some Julia code which catches a parser error and creates a link which can be followed from the terminal, allowing for richer formatting and whatever browser goodness. (In this demo, it's not actually richer because I just sent along the ANSI-formatted error string. But we could send html or all sorts of things in the query payload.) julia> using JuliaSyntax, URIs
julia> try
JuliaSyntax.parse(JuliaSyntax.SyntaxNode, "function fx()\n (x + \nend")
catch exc
iob = IOBuffer()
showerror(IOContext(iob, :color=>true), exc)
str = String(take!(iob))
println(stdout, "ERROR:")
println(stdout, str)
println(stdout, "\e]8;;$(URI(scheme="file", path="/home/c42f/error_test.html", query=Dict("ansi"=>str)))\e\\<open error in browser>\e]8;;\e\\\n")
end |
|
Nice!
I'm sure it would even be possible to open that in VSCode's built-in web view, which could lead to some interesting workflows. Btw, do you know if there's a way to detect if the terminal supports that ANSI link? When I tried in in the built-in terminal inside VSCode using git-bash it just printed the string but it wasn't clickable.
|
|
btw, I tested that copy pasting from the HTML example you shared works with Outlook and GMail (i.e. color from syntax highlighting is preserved), but does not work with Slack for example. What do you think about still using minimal ascii art for conveying some important information? Or maybe ascii art if the line is pure ascii? |
Auto detection is very uneven, unfortunately. Some terminals do support this and I think one of the most sane is kitty: https://sw.kovidgoyal.net/kitty/kittens/query_terminal/ There's the ancient terminfo database, but I very much doubt that has info on hyperlinks which are a recent feature. There's the TERM environment variable which can sometimes be useful, I don't know? Generally terminal feature detection is a bit of a disaster area. Some discussion can be found here: IIUC the hyperlink escape sequence is designed to just be ignored by most terminals which don't support it. And it's actually enjoyed quite wide and rapid implementation across terminal emulators, as described in https://gist.github.com/egmontkob/eb114294efbcd5adb1944c9f3cb5feda So I think there's a good chance we could turn it on by default. But I'm fairly sure we can't rely on it yet - need several years for wide deployment I suspect.
IIUC this should probably work according to microsoft/vscode#39278 ? I don't know what's going wrong.
Yes. And similarly there's no way to get it to work in github markdown. Because after all these years github still doesn't have any way to get colors into markdown. Though... we could solve that with a "copy as image" done on the javascript side. Or similarly "copy as markdown" which could include alt-text for some kind of selectability? |
Ah no, but that wouldn't be able to be pasted anywhere as a single clipboard entity. "Copy as image" is probably the best we can do :-/
Yes I suspect we will need this in some form. Unsure whether there's a way to have it made optional. Needs experimentation.
No, we need the mechanism to work for unicode. A big fraction of mathematical Julia users love their unicode :-) The |
|
Anecdotal but I've had people tell me "please share stacktraces as text instead of images", probably because it is lesser bandwidth and it is copy-paste / search friendly. |
Ah yes, I forgot about those pesky mathematicians :) Yes, we'll have to support Unicode and I agree, I think using |
|
Do we want to start by collecting and categorizing error messages into some kind of database? A GitHub issue? Did you have something else in mind? Then we can start iterating on best ways to visually depict them / what hints we should provide. |
Absolutely, I do dislike images in many cases. So it leaves us in an annoying situation that (a) many platforms can't deal with anything but text... but (b) text is a poor lowest common denominator. Actually hmm. What if the error page had a corresponding public URL, and there was a "share" button (instead of copy as html?) Then users could share the "public error URL" which would include the rich error information in the query string and can be displayed client-side. People who the link is shared with can just click on that URL to open in browser themselves.
I don't have something in mind yet. I think summarizing them all in a github issue would be fine for now! It's quite easy to find the existing error messages in parser.jl. Here's a prototypical example Line 1859 in 1e12afe |
c42f
changed the title
|
I think that's an interesting idea. It could even be a statically hosted page on for example github pages, right? It would require access to the internet to work, but if the purpose is to make it "rich" when people are sharing that's probably fine too. I think the main downside I see is when people are working in HPC environments, long running background jobs with stdout and stderr to log file, or on CI or in remote sessions where this ANSI link doesn't work. We'll have to display a long URL in those instances, right? |
Definitely. Any dynamic stuff can be done client-side in javascript. Having a static server is so much easier. I was imagining that the link would point to a
Maybe, but we'd still have the normal plain text based error reporting for these cases. The URL idea is just a way to give a richer more interactive experience - to upgrade the user experience for people who use terminals. As an optional add on, perhaps we shouldn't get too distracted by this idea 😅 I'm definitely not wanting to give up on good text-based error reporting. There's a few cases
|
|
I've been working on improving the highlighting of source ranges in the terminal. I've been through a lot of iterations at this point and I do think using the box drawing characters from WGL4 makes sense, they're a good balance of:
Sticking to WGL4 would mean avoiding the cute box drawing characters with rounded corners. But I think that's no great loss and I've observed less than perfect font support for those. (eg, github, rendered on linux with whatever fonts I have locally) I've also found it's helpful to prepend any lines of annotation with a Julia I'd like to keep error messages taking the minimal vertical space relative to the code. Here's a simple example of what I've implemented so far: julia> (x - (c <--- d))
ERROR: ParseError:
(x - (c <--- d))
# └──┘ ── invalid operator @ REPL[48]:1:9And with color
Sometimes multiple errors occur in the same place. Here's an aspirational mock up of what it might look like if we coalesce error printing so that a source line is only ever printed once (ignore the source ranges and precise error messages. This is just a mockup.) (( * b ) - (c -- d) - )
# └──┬──┘ └┤ │
# └─────────────────── not a unary operator @ REPL[35]:1:4
# └──────── invalid operator @ REPL[35]:1:15
# └─ unexpected closing token @ REPL[35]:1:23Or, with color,
I'll probably post a PR of a simple version of this soon. It won't be super fancy (I probably won't do the merging of multiple errors), but hopefully will be a nice improvement on what we have so far. |
|
Wow! I think they are fancy enough and super helpful to better understand errors. Thank you for taking your time to improve this feature! |
|
Glad you like it so far ❤️ Here's some more examples of source range highlighting for cases which span multiple lines. These examples are taken from some testing I was doing related to #212. # -----------------------------------------------------
# BesselK_0.1.0/paperscripts/testing/matern_derivative_speed.jl
const zz = @SVector zeros(2)
# ┌───────────────────────────
const TESTING_PARAMS = (@SVector [1.25, 0.25, 0.5],
@SVector [1.25, 5.25, 0.5],
@SVector [1.25, 0.25, 0.75],
@SVector [1.25, 5.25, 0.75],
@SVector [1.25, 0.25, 1.0],
@SVector [1.25, 5.25, 1.0],
@SVector [1.25, 0.25, 1.75],
@SVector [1.25, 5.25, 1.75],
@SVector [1.25, 0.25, 3.0],
@SVector [1.25, 5.25, 3.0],
@SVector [1.25, 0.25, 4.75],
@SVector [1.25, 5.25, 4.75])
#──────────────────────────────────────────────────┘
# -----------------------------------------------------
# Blobs_0.5.1/src/vector.jl
@inline function Base.iterate(blob::BlobVector, i=1)
(i % UInt) - 1 < length(blob) ? (@inbounds blob[i], i + 1) : nothing
# └────────────────────────┘
# -----------------------------------------------------
# BridgeSDEInference_0.3.2/src/solvers/tsit5.jl
function tsit5(f, t, y, dt, P, tableau)
# ┌──────────────────────────────────────────────────────────────────────────
(@unpack c₁,c₂,c₃,c₄,c₅,c₆,a₂₁,a₃₁,a₃₂,a₄₁,a₄₂,a₄₃,a₅₁,a₅₂,a₅₃,a₅₄,a₆₁,a₆₂,
a₆₃,a₆₄,a₆₅,a₇₁,a₇₂,a₇₃,a₇₄,a₇₅,a₇₆ = tableau)
#─────────────────────────────────────────────────────────┘ |
|
PR for improving this is at #215. Gosh this one felt like it was more work than it appears from the amount of code there. |
Hi,
This is not really an issue, but a couple of projects I found could be helpful and inspire features. I came across this https://github.com/zesterer/chumsky and this https://github.com/zesterer/ariadne Rust projects for parsing compiler errors and got nice diagnostic renderings, I don’t really know if there is an overlap with the JuliaSyntax.jl project, but thought might spark others ideas 😃
PD: I was encouraged to post this from a Slack channel.
The text was updated successfully, but these errors were encountered: