Explore ways to identify changes in hidden elements, like menus

[Mr0grog]

A common problem analysts run into is changes to parts of the page that are not visible by default, such as expanding/popup menus. What are some ways to we can aid analysts in discovering those without sending them to a diff of HTML source code they may not be able to read?

Some example changes, with hidden changes in flyout menus:

• https://monitoring.envirodatagov.org/page/b5df659f-e1eb-400f-bcf3-22567c227f4c/a0ca5b6f-be25-4fbe-b5c9-7298dbfd3f57..51aba759-3bb2-4022-9704-4965ea4d6c74
• https://monitoring.envirodatagov.org/page/750b509d-5e62-44b3-896e-ce2e680d3d23/162f59cb-b743-4951-aba3-12157f857c43..c9881ff1-a624-45e7-aa68-1a7276262ccb

If we can identify a change in the DOM, we can roughly determine whether it’s displayed based on whether it has a width and/or height (similar to jQuery’s :visible selector). One approach might be walking up the tree from any invisible changes to find the deepest one that is visible, then highlighting that with something indicating that it contains a hidden change. It’s not perfect—we can’t say where to click (or whatever) to display the changed content—but it might still be a useful hint.

Any other brilliant ideas for this?

---

Edited 2019-09-16 to update example links.

[Mr0grog]

Definitely still highly relevant.

[lightandluck]

Added never-stale label

[Mr0grog]

While working on a different problem this afternoon, I hacked together a quick script for identifying elements with hidden content that could be used for this. It just draws a little orange dot on the top right corner of any elements with hidden children:

// Yield elements inside `root` (and including `root`) that are invisible
// (offscreen, transparent, etc.). It doesn't descend *into* the invisible
// elements, though.
function *findInvisibles (root, context = null) {
    const computed = getComputedStyle(root);
    if (computed.opacity === '0' || computed.visibility === 'hidden' || computed.display === 'none') {
        yield root;
        return;
    }
    if (!context) {
        context = {
            bodyTop: document.body.getBoundingClientRect().top
        };
    }
    const bounds = root.getBoundingClientRect();
    if (bounds.right < 0 || bounds.bottom < context.bodyTop || (bounds.height === 0 && computed.overflow === 'hidden')) {
        yield root;
        return;
    }

    const children = root.childNodes;
    const childCount = children.length;
    let hasChildren = false;
    let hasVisibleChildren = false;
    let invisibleChildren = []
    for (let i = 0; i < childCount; i++) {
        if (children[i].nodeType === Node.ELEMENT_NODE) {
            hasChildren = true;
            let innerInvisibles = findInvisibles(children[i], context);
            if (!hasVisibleChildren) {
                innerInvisibles = [...innerInvisibles];
                if (innerInvisibles[0] !== children[i]) {
                    hasVisibleChildren = true;
                    for (let invisible of invisibleChildren) yield invisible;
                }
                else {
                    invisibleChildren = invisibleChildren.concat(innerInvisibles);
                }
            }
            if (hasVisibleChildren) {
                for (let invisible of innerInvisibles) yield invisible;
            }
        }
        else if (children[i].nodeType === Node.TEXT_NODE || children[i].nodeType === Node.CDATA_SECTION_NODE) {
            if (children[i].textContent.trim()) hasVisibleChildren = true;
        }
    }
    if (hasChildren && !hasVisibleChildren) yield root;
}

// Draw an orange dot on the top-right corner of an element. The dot is
// a DOM element that lives in a body-level container (so you can easily
// manipulate all the dots together). Dots aren't hosted inside the
// element they mark so we don't have to worry about whether the element
// is a containing block (or changes between being a containing block
// and not being one).
function makeDot(element) {
    var pageBounds = document.body.getBoundingClientRect();
    var elementBounds = element.getBoundingClientRect();

    var dot = document.createElement('div');
    dot.className = 'invisble-change-indicator';
    Object.assign(dot.style, {
        boxSizing: 'border-box',
        backgroundColor: 'orange',
        borderRadius: '7px',
        border: '2px solid white',
        width: '14px',
        height: '14px',
        position: 'absolute',
        left: `${elementBounds.right - 19 - pageBounds.left}px`,
        top: `${elementBounds.top + 5 - pageBounds.top}px`,
        zIndex: '9999999999'
    });

    var dotBox = document.getElementById('wm-diff-dot-box');
    if (!dotBox) {
        dotBox = document.createElement('div');
        dotBox.id = 'wm-diff-dot-box';
        document.body.appendChild(dotBox);
    }
    dotBox.appendChild(dot);
}

// Find all the elements that have invisible elements inside them
invisibleParents = new Set();
for (let invisible of findInvisibles(document.body)) {
    // We only care if the invisible area is or contains our
    // diff insertions/deletions
    if (invisible.querySelector('.wm-diff') && !invisibleParents.has(invisible.parentNode)) {
        invisibleParents.add(invisible.parentNode);
        makeDot(invisible.parentNode);
    }
}

[Mr0grog]

What probably should happen from here:

• Figure out how to best visually indicate the hidden change (this orange dot is probably not clear enough; we probably want some kind of icon, maybe with a tooltip on hover).

• Write a transform (see https://github.com/edgi-govdata-archiving/web-monitoring-ui/blob/master/src/scripts/html-transforms.js) that injects a script like the above (but with better icons or tooltips or whatever) to run after the load event. (Or at some other similar time — we need to wait until CSS has loaded on the page to determine what is/isn't visible.)

  We should probably only apply this transform if the removeFormatting setting is false. (Nothing should be hidden if removeFormatting is true, so this would be pointless then.)

• Possibly add some setting (like the “remove formatting” checkbox) to turn this on/off.

[SYU15]

This sounds super useful. Mind adding a screenshot to show what it looks like now?

[Mr0grog]

Quick screencap from throwing the above script in the web inspector:

[Mr0grog]

Another note on the above as to why this isn’t perfect: there is an element that changed in the container that holds the whole navigation bar, and it looks like no interactions on the page will ever make that element visible. So the orange dot over “Mission” in the menu is actually for the whole menu, and indicates a change you can never actually see no matter what you do. I’m not sure there’s any feasible way to determine whether an invisible change might or might not be revealable.

For reference, the change above is: https://monitoring.envirodatagov.org/page/9328fdfb-f8ad-4212-a638-b0782cf83bfb/53ac6d02-917d-499d-82fb-7b2e643a7866..d4c76163-3cf5-4efd-a16d-bb7b27eebb28