How to scale properly world to canvas

[mwpowellhtx]

Hello,

I'm having what I think is an odd precision issue converting from world coords to canvas bitmap.

Assumptions. The canvas bitmap has a resolution, presumably some pixels width and height, int. Although from a canvas perspective, these are, I think, in terms of float, correct? That's first.

Second. world coordinates are intentionally tracked in decimal, because precision is critical. Measurements are to the centimeter whole number, including fractional decimal places for smaller than that. So cannot have float, or especially double, floating point rounding precision errors going on.

Third. World true zero is not canvas {0,0} presumably top left, usually. There is a simple offset {dX,dY} getting to a zero world coordinates that is aligned with zero canvas. Neither here nor there, I think, for purposes of discussion, just an interesting translational mad fact.

Anyway, at the moment, I have this translation going on from Location (3D, including Z, but we do not care about that dimension, decimal) to SKPoint (float). Which I feel is scaling in correctly, which I suspect may be due to some precision issues between decimal through int to float or some parts in between.

The basic algorithm is to assess the ratio of world coordinate in world dimensions, and apply that into the canvas bitmap dimension. The fun part is, the algorithms and I think scaling to SKIA points is at least consistent. But the location I can assure you, precision wise, is off.

public static SKPoint ToSkiaPoint(this SKBitmap bitmap, Location position)
{
    var worldToCanvasOffset = WorldToCanvasZeroOffsetCentimeters;

    var canvasPos = position + worldToCanvasOffset;

    // Take only two leaving out the Z coord.
    var coords = canvasPos.Coordinates.Take(2).Select(_ => _.Value).ToArray();

    // We need the precision in terms of defimal or even float at this point.
    (decimal bwidth, decimal bheight) = (bitmap.Width, bitmap.Height);
    (decimal wwidth, decimal wheight) = WorldSizeCentimeters;

    // Here 'last' is not Z but rather the Y coord.
    (decimal x, decimal y) = (coords.First() / wwidth * bwidth, coords.Last() / wheight * bheight);

    return new((float)x, (float)y);
}

So the algo goes something like:

1. Translate position to canvas friendly zero {0,0}.
2. Take the first two {X,Y} coordinates from the position.
3. Identify the bitmap and world width and height components.
4. Apply the appoprirate ratio to each of the coordinates. Ratio should be generally, "bitmap dims per world dims".
5. This should be our SKPoint.

I think maybe I see the issue, possibly. Perhaps I need for all the components to land in float, for smoothest possibly scaling into canvas SKPoint.

My next steps are to decompose the calculation a bit and better assess things. But if there are any tips how to perform this calculation more precisely, I am all ears.

[mwpowellhtx]

Although I will say after reading couple of background blogs, maybe there is indeed an easier way to train the canvas as to the world coordinates, and let it handle those translations and scalings for us; which would be beneficial, because then, potentially, we could simply make the direct jump from our world coordinates straight through the canvas. POTENTIALLY...

[mwpowellhtx]

I should probably also mention, unlike the academic scaling transformative examples, in which simple shapes are drawn on a blank canvas, I am loading a reference file from drive, on which I want to apply some scaled (to that bitmap resolution) area, translated then scaled from world coordinates. That's a pretty non-trivial difference, IMO, and something that should be taken into consideration. The reference file is a "hard fact" non-negotiable in that equation. Without which the properly translated and scaled layers above which are meaningless.

In very high level 10K meter perspective; I am adding layers to a map; in other words, it is a cartography problem. And the map, although more or less fictional in nature, is a hard fact about that problem.

Hope that helps, if any one is thinking of making any suggestions toward solving the present dilemma.

Cheers, thanks... 🍻

[mwpowellhtx]

I adjusted the algo slightly, but still appears to be a decimal to float precision issue going on scaling into the int based canvas bitmap.

// SKPointM is decimal based, we do the world-to-canvas translation shifting canvas friendly {0,0} zero.
var pointM = position.ToPointM();
var (px, py) = ((float)pointM.X, (float)pointM.Y);

// We need the precision in terms of defimal or even float at this point.
var (bwidth, bheight) = (bitmap.Width, bitmap.Height);
//(decimal bwidth, decimal bheight) = (bitmap.Width, bitmap.Height);
var (wwidth, wheight) = ((float)WorldSizeCentimeters.Width, (float)WorldSizeCentimeters.Height);
//(decimal wwidth, decimal wheight) = WorldSizeCentimeters;

// Here 'last' is not Z but rather the Y coord.
var (x, y) = (px * (bwidth / wwidth), py * (bheight / wheight));
// Best if thought of as "times 'bitmap per world'" units, cancels the world units, should be landing in bitmap units.

return new(x, y);

The values in question in the debugger.

(float px, float wwidth, int bwidth, float bitmapPerWorldX, float x) = (451915.531, 1500000, 16384, 0.0109226666, 4936.12256);
(float py, float wheight, int bheight, float bitmapPerWorldY, float y) = (993356.9, 1500000, 16384, 0.0109226666, 10850.1064);

Our baseline cartography bitmap is 16384x resolution. As stated earlier, that is a "hard fact", and whatever else we do, that scale cannot be mutated.

The world coordinate system is 1500000x decimal, that, too is a fact, but we want to precisely fit the location into the overlay.

The strategy is to apply the "bitmap per world" unit conversion, which we think should be canceling out the world, and arriving in terms of (hopefully precise) bitmap canvas positions.

I've tried to calculate with decimal through and through, convert to float at the very end, no difference. Or float through and through. No difference, still there is some sort of scaling issue going on.

I'm almost wondering is there an aspect ratio thing going on, but that does not seem quite as likely.

[mwpowellhtx]

So far what I have tried...

1. Do calculations in terms of decimal until the last possible (float)coord moment.
2. Do calculations in terms of float.
3. Do calculations in terms of double until the last possible (float)coord moment.

None of which seem to be making the least bit of difference. The core issue, is that I want to cram a large world coordinate space very precisely into a smaller bitmap resolution.

The next thing I will try for is to perhaps start with a larger resolution source bitmap, which I have several, one in particular extremely large, but really, for performance issues, trying to avoid that if I can help it.

[mwpowellhtx]

Next tried loading from higher resolution baseline image, 32Kx, does not improve things.

Will investigate the efficacy scaling the baseline image out to 1:1 world:canvas if at all possible. And maintain a reference to that baseline bitmap. Might also help improve overall performance not having to load the file every time. Depending what SKIA does expanding the bitmap into a larger bitmap canvas resolution.

[ojb500]

I can't see any big problem with what you are doing. What is going wrong exactly?

I wouldn't worry too much about float precision, if it is a real concern then transform your coordinates so they all lie near 0 where float has maximum dynamic range, but if you're drawing to a bitmap there is a bound on how precise you can possibly be (the equivalent of 1 pixel in world space)

[mwpowellhtx]

There's only so much I can do re: transformations. I do the translation from world zero to canvas zero at the outset, i.e. ToPointM.

In terms of 'world' or more specific 'view ports', yeah, that is another idea. Bit more complex to think about, but certainly another path to consider.

Trying to resize the image itself to anything approaching 1.5M resolution, non-starter, Skia balks at that. Tried even {w*w, h*h}, from 16K res or so, also non-starter Skia exception.

So starting to wonder the view port approach FTW. Then could potentially scale that out to a closer 1:1 world:canvas resolution.

[mwpowellhtx]

To clarify, as the first image suggests, the circle and points are drawn relative to each other in the correct geometries. However, the overall map position is off, as indicated by the arrow, should be shifted somewhat to the left. Which I think is the resolution scaling issue.

[mwpowellhtx]

The very tricky part is that the source image is rotated 180 degrees from what it needs to be in the final product. I suppose could also rotate the view port 180 degrees and do the appropriate view translations accounting for view top left and just do it that way. Then also rotate the view when ready to output, present, etc.

[mwpowellhtx]

Turning the problem over in my mind's eye, still I think there must be some function of the view scaling, world:canvas that can be calculated and translated for. Because any way you slice it, jumping through that first initial bitmap, the view scale there in, there will al ways be that inaccuracy. Does not matter what I am scaling for; rectangle edges i.e. for that view port, circles, points, whatever.

[mwpowellhtx]

I will also add, I also apply layers for grid lines and sector keypad labeling. These are quite precise, but I am also not leaning into the world coordinate system to accomplish that, but rather the bitmap dimensions themselves, and given some forward knowledge on the expected rows and columns in the grid, overlayed keypad, in the appropriate orientations and so forth.

[mwpowellhtx]

Following up, there is this reference in the SKIA docs, but I do not know what the CSharp analog might be, I do not see a SKRect.MakeXYWH, for instance, which to me might see the best ideal letting SKIA do the heavy lifting, if at all possible.

Then there is also this GPT dialog I have been adapting, somewhat, trying to drill into potential problem areas of concern. And while I am loathe to "let" any AI direct my coding, if it can inform me the strongest, most robus algorithm, efficacy, pros cons, then why not.

But still, I am not hundred percent sold I can repeatably determine the post-scaling translation consistently. The chief opportunity is, that whenever I want to jump through the world lens to canvas for any reason, there is always a progressively worser post-scaling tranlation issue.

One case in view, pardoning the pun, from 1.5M world dimensional length, that is a hard constraint, to 8192x bitmap image dimention, effectively about 1/183 scaling world:canvas. Of coruse that ratio can vary with more or less dimensional length; ~64Kx in the most extreme, not taking into consideration load on CPU, RAM, etc.

So, bottom line, I am after the best possible algorithm. Maybe the third exchange with my GPT session is the most promising. The poly-regression approach might be a bit overkill, I'm not sure yet.

Then there is the matter of how I may facilitate such a calculation; which as I understand it System.Numerics matrices might be the way to go there, however, can be tedious to work with matrices at any level.

Anyway 🍔 🌭 🍕 for 🧠 ...

[mwpowellhtx]

Maybe I am doing too much work there as well thinking about it. If I can simple lean into the SKCanvas.Scale method. Are there examples how to use this? That would be grand if that handled the nasty details for us, especially when points are getting less precise because the world coordinate system is larger than the bitmap image resolution. And we could simply concern ourselves with the drawing parts, either in world (ideally) or canvas coordinates. Worse case there is also SKIA matrix support, if necessary, but might like to avoid that scenario.

[mwpowellhtx]

Actually I wonder if that is something else entirely... Like as in it is drawing a scale on the map. Although I do not see any Paint choices there; but which would be great nonetheless; albeit not exactly what I am talking about here.

[mwpowellhtx]

Okay so not drawing a map scale, but from this SKIA fiddle example, it is rather academic, some things need to be properly understood, like is that scale of world:bitmap... so if we know we have 1.5M centimeters in a dimension, and a bitmap resolution of 8192x, then we can say px/centimeter or 1/183, or this ugly floating point value (Windows Calculator) 0.00546448087431693989071038251366 albeit in single precision float, so more like akin to 0.00546f perhaps... which probably accounts for some of the drift I expect to see the further out we go. Even if we set bitmap-center as the scaling center point, maybe that mitigates somewhat. Then do we also need to account for X:Y aspect ratios. Overall the docs are stone cold silent on the Scale method itself.

[mwpowellhtx]

Okay so this thing adapting Scale in, I'm not sure where it is drawing, if at all. I see no evidence that it is. Or that Scale is otherwise being respected throughout, at least not based on the academic fiddle example (above). That's first off.

public static void VerifyCanvasScaleDryRun(string imagePath, Guid uuid = default)
{
    string fullPath;

    if (uuid == default)
    {
        uuid = Guid.NewGuid();
    }

    fullPath = $"Maps\\canvas_scale_dryRun_{uuid = Guid.NewGuid()}.jpg";

    using var image = SKImage.FromEncodedData(imagePath);
    using var bitmap = SKBitmap.FromImage(image);
    using var canvas = new SKCanvas(bitmap);

    var bitmapWidthM = (decimal)bitmap.Width;
    var bitmapWidthF = (float)bitmap.Width;
    var bitmapWidthD = (double)bitmap.Width;
    var scaleM = bitmapWidthM / WorldSizeCentimeters.Width; // 8192 / 150000 in our use case...
    // TODO: this is a problem... Windows Calculator reports: 0.00546448087431693989071038251366
    var scaleF = bitmapWidthF / (float)WorldSizeCentimeters.Width;
    var scaleD = bitmapWidthD / (double)WorldSizeCentimeters.Width;
    // TODO: but single/double are reporting something more like this: 0.0054613333333333337
    canvas.Scale(scaleF);

    // TODO: pg record: {"Uuid":"b8ec5136-c7e8-4e7b-bfc4-ab26d394c8c2","X":-438084.469,"Y":103356.875,"Z":38187.777}
    const float radius = 10000f;
    using var paint = new SKPaint { Color = SKColors.Red.WithAlpha((byte)(byte.MaxValue * 0.5f)) };
    SKPoint point = new(-438084.469f, 103356.875f);
    canvas.DrawCircle(point, radius, paint);
    canvas.Flush();

    const float inflation = radius * 10;

    SKRect rect = new((point.X - radius) * scaleF, (point.Y - radius) * scaleF, (point.X + radius) * scaleF, (point.Y + radius) * scaleF);
    rect.Inflate(inflation * scaleF, inflation * scaleF);

    var bitmapRectI = bitmap.GetBoundsRectI();
    var viewBitmap = new SKBitmap((int)rect.Size.Width, (int)rect.Size.Height);

    using var viewCanvas = new SKCanvas(viewBitmap);
    viewCanvas.DrawBitmap(bitmap, bitmapRectI, viewBitmap.GetBoundsRectI());

    // TODO: canvas scale makes it look very grainy pixelated... not gonna work for us...
    // TODO: image quality is EVERYTHING
    using var data = bitmap.Encode(SKEncodedImageFormat.Jpeg, 100);
    using var fs = new FileStream(fullPath, FileMode.Create, FileAccess.Write, FileShare.Read);
    data.SaveTo(fs);
}

Secondly, there are definitely floating point rounding issues going on. I'm not sure quite what we can do about that, which is going to make dealing with some resolutions very challenging when decimal centimeters accuracy precision is literally EVERYTHING to the cartography question.

Just on the numbers alone, I'm not sure how they get 0.0054613333333333337 from 0.00546448087431693989071038251366 from about the sixth decimal point forward. Maybe there is a precision numerics solution to that, I do not know yet. Or that we might even think about kicking it out to a C++ library where we do have precision, and loading that result. Even taking the raw float numbers, not converted from decimal, the result is not better, testF = 0.00546133332 /* 8192f / 1500000f */.

Drawn with scaling involved, still the same sort of floating point rounding issues evidenced. The only up side is that we can more than less specify geometry in terms that are literally world scale. The following example, not rotated, but still very inaccurate, manifestation of the rounding issues.

There are also issues capturing view ports when scaled, result is VERY pixelated. Maybe need to reset scale to 1f approaching that bit, I do not know. Somewhat separate issue, the rounding issue is the core problem right now.

Will experiment with higher resolution versions of the source map to see if and how it may improve. I would expect some, but kind of at a loss how to achieve accuracy at this point.

[ojb500]

Can you share some of your SKPointM code? decimal may not be a good choice for doing this kind of math.

[mwpowellhtx]

Can you share some of your SKPointM code? decimal may not be a good choice for doing this kind of math.

Basically a float-decimal conversion from the SkiaSharp code. decimal is a must because, technically, notwithstanding dotnet floating point rounding shenanigans, whether in centimeters or meters, precision is a must. Cannot have any rounding issues that would influence the results.

So... whereas decimal is a great fit for financial applications, for instance, because of that. It is also a must have for this one, for the aforementioned reaons. We basically do a float conversion, or BigDecimal, when doing the calculations, as the case may be.

But were also experiencing floating point rounding issues beyond 5-6 places. Troubling to say the least.

[ojb500]

decimal is still a floating point type and you can still accumulate errors if you're not careful - it's just bigger than the others and uses 10 as the base rather than 2. It's very good at representing decimals but might not be as good as double for some cases (see what happens when you do Console.WriteLine(1M / 3M * 3M); vs Console.WriteLine(1.0 / 3.0 * 3.0);)

When you calculate your scaling factor this should be done with fairly wide precision (probably double is more than adequate as the number will be small) and the subsequent multiplication of the coordinates should also be done in wide precision. Only then (when you're dealing with pixel positions, small-ish numbers between 0-8192) should you convert your coordinates to floats and make an SKPoint. A float has no problem representing any integer up to about 16 million or so.

[ojb500]

See this fiddle - decimal seems to give the most significant digits in the scaling factor, but leads to a surprising result.

[mwpowellhtx]

An interesting observation but for different reasons, you are verifying what we were seeing in the CSharp discords earlier.

Float scaling factor: 0.0054613333
Double scaling factor: 0.005461333333333334
Decimal scaling factor: 0.0054613333333333333333333333

All things considered, maybe the scaling factor "is fine".

But actually, I think the issue may be rooted in the source coordinates themselves, as noted above.

Unconfirmed, but the thing reporting the coordinates, I'm not sure may be truncating or rounding itself, and maintaining more precise numbers internally. Although I cannot be certain. When I do the math starting from an image editor, and ratios on screen, then the result is what is expected.