RFC: use IOContext more consistently. part of #14052 #16354

JeffBezanson · 2016-05-13T20:44:25Z

The whole output thing is difficult to wrap my head around, but I think I found a nice chunk to bite off.

First, showarray and showdict were undermining IOContext, so they are removed, making the IOContext flags the only thing that controls printing. That fixes e.g. #13710. It might also fix #16103 and #16267.

Second, some definitions of writemime were using it to mean "how to print in the REPL", which is not correct. Those needed to be removed in preparation for merging show and writemime. Instead I added an :interactive flag, which picks between e.g. multi-line and single-line display of vectors. REPL display is now done properly, by having the REPL print to an IOContext that sets :limit_output and :interactive.

The next steps are (1) fully merge show and writemime, (2) decide exactly which output flags we need. (2) is harder than I thought. The important variables currently seem to be:

Whether collections are truncated
Whether to use a more compact single-line format, or something better for interactive use
How many digits of numbers to print, and whether to use the print_shortest algorithm
How does summary fit in? Maybe it doesn't, and can stay a separate function.

JeffBezanson · 2016-05-13T20:48:59Z

I think this also might fix #6117, since show methods for new array types will get called now.

JeffBezanson · 2016-05-13T21:03:26Z

I see we are currently using limit_output to print shorter numbers. I don't think that's right --- it seems to me truncating containers is a different issue than how many digits to print.

vtjnash · 2016-05-13T21:15:41Z

Biking-shedding interactive, maybe this should be pager? I'm not sure we want to encourage branching on whether the immediate display is "interactive". For example, I think Juypter may want the entire output since it implements it's own pager (aka a scrollbar).

JeffBezanson · 2016-05-13T21:18:42Z

Yeah, "interactive" may not be the right concept. We can certainly rename it.

Jupyter can still turn off limit_output though; that's orthogonal.

nalimilan · 2016-05-13T21:26:26Z

Couldn't that interactive/pager information be obtained via displaysize? I guess a display which doesn't require wrapping lines should return (0, 0).

Regarding other variables, it seems that an important issue is whether to print type information or not. The typical use case for that is arrays which can print the type only once, and then use the most compact output possible for elements, vs. printing them individually. Cf. discussion after #14052 (comment) and #15963.

JeffBezanson · 2016-05-13T21:29:28Z

To clarify, the :interactive flag in this PR distinguishes the following 2 output formats:

julia> rand(5)
5-element Array{Float64,1}:
 0.410296 
 0.259393 
 0.0694566
 0.60467  
 0.433025 

julia> show(rand(5))
[0.46591788434697134,0.8616631147875908,0.2990903979811059,0.3327958869667973,0.04755225542278496]

toivoh · 2016-05-14T05:10:30Z

show_unquoted passes around an indent value to be able to pretty print expressions. Perhaps this could eventually be put into IOContext? I think there's plenty more cases where indenting would be useful to make readable printouts, eg nested data structures.

JeffBezanson · 2016-05-17T13:34:02Z

@vtjnash and I were discussing that perhaps :interactive should be :multiline, to distinguish single- from multi-line ways to show data structures. This seems like a useful distinction to have.

JeffBezanson · 2016-05-20T03:58:02Z

Ok, I think things are pretty clean here now. I used the following 3 output flags:

limit: Omit elements of big containers
compact: Print numbers with fewer digits
multiline: Request interactive-style long-form output, otherwise try to keep output on a single line

showcompact now only has one definition, which just sets compact to true. All types define only show (and summary).

The flags are used in several ways. For example, array elements are always printed with multiline=false, but compact=true is set only if the io object doesn't already have a setting for compact. This lets you get many combinations of output options from the repl by calling show with an appropriate IOContext.

vtjnash · 2016-05-20T04:14:19Z

base/Enums.jl

@@ -93,7 +93,7 @@ macro enum(T,syms...)
            end
        end
        function Base.show(io::IO,x::$(esc(typename)))
-            if Base.limit_output(io)
+            if get(io, :compact, false)


I had been writing these as get(io, :compact, false) === true for future-compatibility / robustness against bad IOContext settings. Thoughts?

I'd rather get an error when a non-Bool is passed in the IOContext than a silent fallback to false. This makes it easier to detect mistakes.

nalimilan · 2016-05-20T08:33:17Z

+1 for multiline.

Though don't we want another flag to set the amount of information to print (and not only number of digits)? Cf. my comment above. @JeffBezanson, you said at #15454 (comment) that you wanted to support several verbosity levels, which was needed for enums printing (but also e.g. for Nullable). Is that still needed?

JeffBezanson · 2016-05-20T15:25:41Z

I think it turns out that most verbosity issues need to be factored into separate flags. I don't have a concrete proposal for a verbosity enum, but we can discuss if somebody puts one forward.

JeffBezanson · 2016-05-20T19:49:55Z

Ok this should be in pretty good shape now.

kmsquire · 2016-05-21T00:06:50Z

@vtjnash https://github.com/vtjnash & I just came to the conclusion
that IOContext is not really the right tool for detailed number formatting.
If you need to control the number of digits, you need something like printf.

For matrix/vector display, having the ability to format all
entries similarly would be nice. Why wouldn't IOContext be good for this?

vtjnash · 2016-05-21T20:16:02Z

fix #16026
fix #15454

JeffBezanson · 2016-05-22T00:08:40Z

having the ability to format all entries similarly would be nice

Yes, this can be debated. Let's focus on just the changes in this PR first.

tkelman · 2016-05-23T09:11:05Z

travis osx failure log backed up to https://gist.github.com/3cd64fa498b4defbd9a9d72198ddf477, restarted

We had several `writemime` definitions whose purpose was to control how objects should be printed in the REPL. Instead this introduces a `:multiline` IOContext flag. Use `:compact` IOContext flag to request shorter printing of numbers. Removes `showarray` and `showdict`, allowing IOContext flags to work better recursively. Fixes #13710. Also related: #16103 and #16267.

tkelman · 2016-05-23T23:12:36Z

Should showcompact_lim have a deprecation added in case any packages were using it? The deprecation for with_output_limit still doesn't appear to do any limiting.

JeffBezanson · 2016-05-24T15:14:44Z

with_output_limit is incompatible with the way this works now. The API is get(io, :limit, default), so you either get the setting from io or default, and there's nowhere to use the global variable. If the deprecation was made to work, then behavior of code using the new API would change when the deprecation is removed. That didn't seem like a good tradeoff to me for a non-exported function.

EDIT: It can mostly work if the global flag is 3-valued.

some small tweaks to output function changes and doc updates

JeffBezanson · 2016-05-24T16:59:59Z

I will merge this after CI. showcompact_lim has been around a pretty short time, but if it turns out to have users we can add a deprecation later.

JeffBezanson · 2016-05-24T17:22:18Z

That AV timeout again.

nalimilan · 2016-05-24T22:04:33Z

FWIW, this (combined with previous PRs) allowed getting rid of about 300 lines of custom printing methods in NullableArrays.jl. Now everything works fine with the default AbstractArray methods!

tkelman · 2016-05-25T15:19:16Z

BlockArrays, CSV, DataStreams, and TypedTables were all using showcompact_lim. Possibly more being obscured by existing failures on 0.5. http://pkg.julialang.org/pulse.html

stevengj · 2016-09-13T01:23:48Z

base/multimedia.jl

@@ -57,31 +57,33 @@ mimewritable(m::AbstractString, x) = mimewritable(MIME(m), x)
 # format and is returned unmodified.  This is useful so that raw data can be
 # passed to display(m::MIME, x).

+verbose_writemime(io, m, x) = writemime(IOContext(io,multiline=true,limit=false), m, x)


Note that this led to JuliaLang/IJulia.jl#462

It's not a big deal; in hindsight, I'm not sure the whole stringmime function belongs in Base anyway. It should be easy enough to implement a limitstringmime in IJulia.

JeffBezanson force-pushed the jb/output branch 2 times, most recently from 1851ede to 08a2841 Compare May 20, 2016 03:22

vtjnash reviewed May 20, 2016
View reviewed changes

JeffBezanson force-pushed the jb/output branch 2 times, most recently from 2e44734 to 8005c0f Compare May 22, 2016 02:13

nalimilan mentioned this pull request May 22, 2016

Update and finetune the output of commands in the latest doc #16511

Merged

nalimilan mentioned this pull request May 23, 2016

Set notebook to certain amount of DP #16522

Closed

JeffBezanson force-pushed the jb/output branch from 8005c0f to b479254 Compare May 23, 2016 16:37

JeffBezanson force-pushed the jb/output branch from b479254 to 5bfe0cf Compare May 23, 2016 20:56

JeffBezanson added 2 commits May 24, 2016 11:41

remove limit_output function, rename :limit_output to :limit

5d7dd9c

use multiline=true, limit=false in reprmime and stringmime

0c3147e

some small tweaks to output function changes and doc updates

JeffBezanson force-pushed the jb/output branch from 5bfe0cf to 0c3147e Compare May 24, 2016 15:41

JeffBezanson merged commit c43b5e5 into master May 24, 2016

This was referenced May 24, 2016

array display is a mess #6117

Closed

print() vs. show() for enums #15454

Closed

REPL defaults to limiting output for new types even though limit_output(STDOUT) is false #16026

Closed

tkelman deleted the jb/output branch May 24, 2016 20:26

tkelman mentioned this pull request May 26, 2016

Julep: extended proposal for fixing show, print, & friends #14052

Closed

jrevels added a commit to JuliaCI/BenchmarkTools.jl that referenced this pull request May 26, 2016

fix show methods in light of JuliaLang/julia#16354

627fa75

jrevels added a commit to JuliaCI/BenchmarkTools.jl that referenced this pull request May 30, 2016

fix show methods in light of JuliaLang/julia#16354 (#9)

fb33ddf

stevengj reviewed Sep 13, 2016
View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

RFC: use IOContext more consistently. part of #14052 #16354

RFC: use IOContext more consistently. part of #14052 #16354

JeffBezanson commented May 13, 2016

JeffBezanson commented May 13, 2016

JeffBezanson commented May 13, 2016

vtjnash commented May 13, 2016

JeffBezanson commented May 13, 2016

nalimilan commented May 13, 2016

JeffBezanson commented May 13, 2016

toivoh commented May 14, 2016

JeffBezanson commented May 17, 2016

JeffBezanson commented May 20, 2016

vtjnash May 20, 2016

nalimilan May 20, 2016

nalimilan commented May 20, 2016

JeffBezanson commented May 20, 2016

JeffBezanson commented May 20, 2016

kmsquire commented May 21, 2016

vtjnash commented May 21, 2016

JeffBezanson commented May 22, 2016

tkelman commented May 23, 2016

tkelman commented May 23, 2016

JeffBezanson commented May 24, 2016 •

edited

Loading

JeffBezanson commented May 24, 2016

JeffBezanson commented May 24, 2016

nalimilan commented May 24, 2016

tkelman commented May 25, 2016

stevengj Sep 13, 2016

RFC: use IOContext more consistently. part of #14052 #16354

RFC: use IOContext more consistently. part of #14052 #16354

Conversation

JeffBezanson commented May 13, 2016

JeffBezanson commented May 13, 2016

JeffBezanson commented May 13, 2016

vtjnash commented May 13, 2016

JeffBezanson commented May 13, 2016

nalimilan commented May 13, 2016

JeffBezanson commented May 13, 2016

toivoh commented May 14, 2016

JeffBezanson commented May 17, 2016

JeffBezanson commented May 20, 2016

vtjnash May 20, 2016

Choose a reason for hiding this comment

nalimilan May 20, 2016

Choose a reason for hiding this comment

nalimilan commented May 20, 2016

JeffBezanson commented May 20, 2016

JeffBezanson commented May 20, 2016

kmsquire commented May 21, 2016

vtjnash commented May 21, 2016

JeffBezanson commented May 22, 2016

tkelman commented May 23, 2016

tkelman commented May 23, 2016

JeffBezanson commented May 24, 2016 • edited Loading

JeffBezanson commented May 24, 2016

JeffBezanson commented May 24, 2016

nalimilan commented May 24, 2016

tkelman commented May 25, 2016

stevengj Sep 13, 2016

Choose a reason for hiding this comment

JeffBezanson commented May 24, 2016 •

edited

Loading