Error if user in no python utf8 encoding ENV to run blazor AOT #53367

newbe36524 · 2021-05-26T03:23:12Z

Solution , click here #53367 (comment)

Describe the bug

User have to set default encoding of python to utf8 when run blazor AOT.

To Reproduce

Try AOT as docs mentioned in a Chinese locals PC

https://devblogs.microsoft.com/aspnet/asp-net-core-updates-in-net-6-preview-4/#blazor-webassembly-ahead-of-time-aot-compilation

error as below:

Compiling native assets with emcc. This may take a while ...
  Traceback (most recent call last):
    File "C:\Program Files\dotnet\packs\Microsoft.NET.Runtime.Emscripten.2.0.12.Sdk.win-x64\6.0.0-preview.4.21220.1\tools\emscripten\emcc.py", line 3315, in <module>
      sys.exit(main(sys.argv))
    File "C:\Program Files\dotnet\packs\Microsoft.NET.Runtime.Emscripten.2.0.12.Sdk.win-x64\6.0.0-preview.4.21220.1\tools\emscripten\emcc.py", line 3308, in main
      ret = run(args)
    File "C:\Program Files\dotnet\packs\Microsoft.NET.Runtime.Emscripten.2.0.12.Sdk.win-x64\6.0.0-preview.4.21220.1\tools\emscripten\emcc.py", line 2156, in run
      post_link(options, wasm_target, wasm_target, target)
    File "C:\Program Files\dotnet\packs\Microsoft.NET.Runtime.Emscripten.2.0.12.Sdk.win-x64\6.0.0-preview.4.21220.1\tools\emscripten\emcc.py", line 2185, in post_link
      emscripten.run(in_wasm, wasm_target, final_js, memfile)
    File "C:\Program Files\dotnet\packs\Microsoft.NET.Runtime.Emscripten.2.0.12.Sdk.win-x64\6.0.0-preview.4.21220.1\tools\emscripten\emscripten.py", line 876, in run
      return temp_files.run_and_clean(lambda: emscript(
    File "C:\Program Files\dotnet\packs\Microsoft.NET.Runtime.Emscripten.2.0.12.Sdk.win-x64\6.0.0-preview.4.21220.1\tools\emscripten\tools\tempfiles.py", line 104, in run_and_clean
      return func()
    File "C:\Program Files\dotnet\packs\Microsoft.NET.Runtime.Emscripten.2.0.12.Sdk.win-x64\6.0.0-preview.4.21220.1\tools\emscripten\emscripten.py", line 876, in <lambda>
      return temp_files.run_and_clean(lambda: emscript(
    File "C:\Program Files\dotnet\packs\Microsoft.NET.Runtime.Emscripten.2.0.12.Sdk.win-x64\6.0.0-preview.4.21220.1\tools\emscripten\emscripten.py", line 331, in emscript
      glue, forwarded_data = compile_settings(temp_files)
    File "C:\Program Files\dotnet\packs\Microsoft.NET.Runtime.Emscripten.2.0.12.Sdk.win-x64\6.0.0-preview.4.21220.1\tools\emscripten\emscripten.py", line 195, in compile_settings
      out = shared.run_js_tool(path_from_root('src', 'compiler.js'),
    File "C:\Program Files\dotnet\packs\Microsoft.NET.Runtime.Emscripten.2.0.12.Sdk.win-x64\6.0.0-preview.4.21220.1\tools\emscripten\tools\shared.py", line 118, in run_js_tool
      return check_call(command, *args, **kw).stdout
    File "C:\Program Files\dotnet\packs\Microsoft.NET.Runtime.Emscripten.2.0.12.Sdk.win-x64\6.0.0-preview.4.21220.1\tools\emscripten\tools\shared.py", line 104, in check_call
      return run_process(cmd, *args, **kw)
    File "C:\Program Files\dotnet\packs\Microsoft.NET.Runtime.Emscripten.2.0.12.Sdk.win-x64\6.0.0-preview.4.21220.1\tools\emscripten\tools\shared.py", line 94, in run_process
      ret = subprocess.run(cmd, check=check, input=input, *args, **kw)
    File "subprocess.py", line 507, in run
    File "subprocess.py", line 1121, in communicate
  UnicodeDecodeError: 'gbk' codec can't decode byte 0x94 in position 325809: illegal multibyte sequence
C:\Program Files\dotnet\packs\Microsoft.NET.Runtime.WebAssembly.Sdk\6.0.0-preview.4.21253.7\Sdk\WasmApp.targets(504,5): error MSB3073: The command "emcc "@D:\Repo\newbe\Newbe.BookmarkManager\src\Newbe.BookmarkManager\obj\Release\net6.0\wasm\emcc-link.rsp"" exited with code 1. [D:\Repo\newbe\Newbe.BookmarkManager\src\Newbe.BookmarkManager\Newbe.BookmarkManager.csproj]

Set env PYTHONUTF8=1.

Everything works.

I think the root cause is the all emscripten scripts has no encoding header.

Further technical details

net6 pre4

SIkebe · 2021-05-26T03:56:23Z

Japanese locale also has this issue. It requires @newbe36524 's workaround to publish without error.

dotnet publish -c Release
.NET 向け Microsoft (R) Build Engine バージョン 16.11.0-preview-21254-21+e73d08c28
Copyright (C) Microsoft Corporation.All rights reserved.

  復元対象のプロジェクトを決定しています...
  復元対象のすべてのプロジェクトは最新です。
  プレビュー版の .NET を使用しています。https://aka.ms/dotnet-core-preview をご覧くだ
さい
  AoTBlazor -> D:\poc\build2021\AoTBlazor\bin\Release\net6.0\AoTBlazor.dll
  AoTBlazor (Blazor output) -> D:\poc\build2021\AoTBlazor\bin\Release\net6.0\wwwroot
  アセンブリのサイズを最適化しています。これにより、アプリの動作が変更される可能性が
あります。公開した後にテストしてください。https://aka.ms/dotnet-illink を参照してくだ
さい
  AOT'ing 30 assemblies
  Compiling native assets with emcc. This may take a while ...
  Traceback (most recent call last):
    File "C:\Program Files\dotnet\packs\Microsoft.NET.Runtime.Emscripten.2.0.12.Sdk.win-x64\6.0.0-preview.4.21220.1\tools\emscripten\emcc.py", line 3315, in <module>     
      sys.exit(main(sys.argv))
    File "C:\Program Files\dotnet\packs\Microsoft.NET.Runtime.Emscripten.2.0.12.Sdk.win-x64\6.0.0-preview.4.21220.1\tools\emscripten\emcc.py", line 3308, in main
      ret = run(args)
    File "C:\Program Files\dotnet\packs\Microsoft.NET.Runtime.Emscripten.2.0.12.Sdk.win-x64\6.0.0-preview.4.21220.1\tools\emscripten\emcc.py", line 2156, in run
      post_link(options, wasm_target, wasm_target, target)
    File "C:\Program Files\dotnet\packs\Microsoft.NET.Runtime.Emscripten.2.0.12.Sdk.win-x64\6.0.0-preview.4.21220.1\tools\emscripten\emcc.py", line 2185, in post_link    
      emscripten.run(in_wasm, wasm_target, final_js, memfile)
    File "C:\Program Files\dotnet\packs\Microsoft.NET.Runtime.Emscripten.2.0.12.Sdk.win-x64\6.0.0-preview.4.21220.1\tools\emscripten\emscripten.py", line 876, in run     
      return temp_files.run_and_clean(lambda: emscript(
    File "C:\Program Files\dotnet\packs\Microsoft.NET.Runtime.Emscripten.2.0.12.Sdk.win-x64\6.0.0-preview.4.21220.1\tools\emscripten\tools\tempfiles.py", line 104, in run_and_clean
      return func()
    File "C:\Program Files\dotnet\packs\Microsoft.NET.Runtime.Emscripten.2.0.12.Sdk.win-x64\6.0.0-preview.4.21220.1\tools\emscripten\emscripten.py", line 876, in <lambda>      return temp_files.run_and_clean(lambda: emscript(
    File "C:\Program Files\dotnet\packs\Microsoft.NET.Runtime.Emscripten.2.0.12.Sdk.win-x64\6.0.0-preview.4.21220.1\tools\emscripten\emscripten.py", line 331, in emscript      glue, forwarded_data = compile_settings(temp_files)
    File "C:\Program Files\dotnet\packs\Microsoft.NET.Runtime.Emscripten.2.0.12.Sdk.win-x64\6.0.0-preview.4.21220.1\tools\emscripten\emscripten.py", line 195, in compile_settings
      out = shared.run_js_tool(path_from_root('src', 'compiler.js'),
    File "C:\Program Files\dotnet\packs\Microsoft.NET.Runtime.Emscripten.2.0.12.Sdk.win-x64\6.0.0-preview.4.21220.1\tools\emscripten\tools\shared.py", line 118, in run_js_tool
      return check_call(command, *args, **kw).stdout
    File "C:\Program Files\dotnet\packs\Microsoft.NET.Runtime.Emscripten.2.0.12.Sdk.win-x64\6.0.0-preview.4.21220.1\tools\emscripten\tools\shared.py", line 104, in check_call
      return run_process(cmd, *args, **kw)
    File "C:\Program Files\dotnet\packs\Microsoft.NET.Runtime.Emscripten.2.0.12.Sdk.win-x64\6.0.0-preview.4.21220.1\tools\emscripten\tools\shared.py", line 94, in run_process
      ret = subprocess.run(cmd, check=check, input=input, *args, **kw)
    File "subprocess.py", line 507, in run
    File "subprocess.py", line 1121, in communicate
  UnicodeDecodeError: 'cp932' codec can't decode byte 0x94 in position 325809: illegal multibyte sequence
C:\Program Files\dotnet\packs\Microsoft.NET.Runtime.WebAssembly.Sdk\6.0.0-preview.4.21253.7\Sdk\WasmApp.targets(504,5): error MSB3073: コマンド "emcc "@D:\poc\build2021\AoTBlazor\obj\Release\net6.0\wasm\emcc-link.rsp"" はコ
ード 1 で終了しました。 [D:\poc\build2021\AoTBlazor\AoTBlazor.csproj]

dotnet-issue-labeler · 2021-05-27T16:40:10Z

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

ghost · 2021-06-01T17:07:42Z

Tagging subscribers to 'arch-wasm': @lewing
See info in area-owners.md if you want to be subscribed.

Issue Details

Describe the bug

User have to set default encoding of python to utf8 when run blazor AOT.

To Reproduce

Try AOT as docs mentioned in a Chinese locals PC

https://devblogs.microsoft.com/aspnet/asp-net-core-updates-in-net-6-preview-4/#blazor-webassembly-ahead-of-time-aot-compilation

error as below:

Compiling native assets with emcc. This may take a while ...
  Traceback (most recent call last):
    File "C:\Program Files\dotnet\packs\Microsoft.NET.Runtime.Emscripten.2.0.12.Sdk.win-x64\6.0.0-preview.4.21220.1\tools\emscripten\emcc.py", line 3315, in <module>
      sys.exit(main(sys.argv))
    File "C:\Program Files\dotnet\packs\Microsoft.NET.Runtime.Emscripten.2.0.12.Sdk.win-x64\6.0.0-preview.4.21220.1\tools\emscripten\emcc.py", line 3308, in main
      ret = run(args)
    File "C:\Program Files\dotnet\packs\Microsoft.NET.Runtime.Emscripten.2.0.12.Sdk.win-x64\6.0.0-preview.4.21220.1\tools\emscripten\emcc.py", line 2156, in run
      post_link(options, wasm_target, wasm_target, target)
    File "C:\Program Files\dotnet\packs\Microsoft.NET.Runtime.Emscripten.2.0.12.Sdk.win-x64\6.0.0-preview.4.21220.1\tools\emscripten\emcc.py", line 2185, in post_link
      emscripten.run(in_wasm, wasm_target, final_js, memfile)
    File "C:\Program Files\dotnet\packs\Microsoft.NET.Runtime.Emscripten.2.0.12.Sdk.win-x64\6.0.0-preview.4.21220.1\tools\emscripten\emscripten.py", line 876, in run
      return temp_files.run_and_clean(lambda: emscript(
    File "C:\Program Files\dotnet\packs\Microsoft.NET.Runtime.Emscripten.2.0.12.Sdk.win-x64\6.0.0-preview.4.21220.1\tools\emscripten\tools\tempfiles.py", line 104, in run_and_clean
      return func()
    File "C:\Program Files\dotnet\packs\Microsoft.NET.Runtime.Emscripten.2.0.12.Sdk.win-x64\6.0.0-preview.4.21220.1\tools\emscripten\emscripten.py", line 876, in <lambda>
      return temp_files.run_and_clean(lambda: emscript(
    File "C:\Program Files\dotnet\packs\Microsoft.NET.Runtime.Emscripten.2.0.12.Sdk.win-x64\6.0.0-preview.4.21220.1\tools\emscripten\emscripten.py", line 331, in emscript
      glue, forwarded_data = compile_settings(temp_files)
    File "C:\Program Files\dotnet\packs\Microsoft.NET.Runtime.Emscripten.2.0.12.Sdk.win-x64\6.0.0-preview.4.21220.1\tools\emscripten\emscripten.py", line 195, in compile_settings
      out = shared.run_js_tool(path_from_root('src', 'compiler.js'),
    File "C:\Program Files\dotnet\packs\Microsoft.NET.Runtime.Emscripten.2.0.12.Sdk.win-x64\6.0.0-preview.4.21220.1\tools\emscripten\tools\shared.py", line 118, in run_js_tool
      return check_call(command, *args, **kw).stdout
    File "C:\Program Files\dotnet\packs\Microsoft.NET.Runtime.Emscripten.2.0.12.Sdk.win-x64\6.0.0-preview.4.21220.1\tools\emscripten\tools\shared.py", line 104, in check_call
      return run_process(cmd, *args, **kw)
    File "C:\Program Files\dotnet\packs\Microsoft.NET.Runtime.Emscripten.2.0.12.Sdk.win-x64\6.0.0-preview.4.21220.1\tools\emscripten\tools\shared.py", line 94, in run_process
      ret = subprocess.run(cmd, check=check, input=input, *args, **kw)
    File "subprocess.py", line 507, in run
    File "subprocess.py", line 1121, in communicate
  UnicodeDecodeError: 'gbk' codec can't decode byte 0x94 in position 325809: illegal multibyte sequence
C:\Program Files\dotnet\packs\Microsoft.NET.Runtime.WebAssembly.Sdk\6.0.0-preview.4.21253.7\Sdk\WasmApp.targets(504,5): error MSB3073: The command "emcc "@D:\Repo\newbe\Newbe.BookmarkManager\src\Newbe.BookmarkManager\obj\Release\net6.0\wasm\emcc-link.rsp"" exited with code 1. [D:\Repo\newbe\Newbe.BookmarkManager\src\Newbe.BookmarkManager\Newbe.BookmarkManager.csproj]

Set env PYTHONUTF8=1.

Everything works.

I think the root cause is the all emscripten scripts has no encoding header.

Further technical details

net6 pre4

Author:	newbe36524
Assignees:	-
Labels:	`arch-wasm`, `area-Codegen-AOT-mono`, `untriaged`
Milestone:	-

Fixes dotnet#53367

qinhuaihe · 2021-06-03T01:53:07Z

@newbe36524 Could you tell me how to Set env PYTHONUTF8=1. ? Thank you.

radical · 2021-06-03T02:24:22Z

Adding this to your project file should do it:

<ItemGroup>
  <EmscriptenEnvVars Include="PYTHONUTF8=1" />
</ItemGroup>

newbe36524 · 2021-06-03T04:03:45Z

@newbe36524 Could you tell me how to Set env PYTHONUTF8=1. ? Thank you.

I make this done in 'Enviroment Variabes' dialog in my Windows PC by add a new System variable

Python defaults to the system charset, and thus expects the files it's reading to match that. But that might not always be true. Eg. system charset=gbk, failing to read utf-8 files. See dotnet/runtime#53367 for the motivating issue

mkArtakMSFT transferred this issue from dotnet/aspnetcore May 27, 2021

dotnet-issue-labeler bot added the untriaged New issue has not been triaged by the area owner label May 27, 2021

jeffschwMSFT added the area-Codegen-AOT-mono label May 28, 2021

lewing added the arch-wasm WebAssembly architecture label Jun 1, 2021

lewing added this to the 6.0.0 milestone Jun 1, 2021

lewing assigned radical Jun 1, 2021

lewing removed the untriaged New issue has not been triaged by the area owner label Jun 1, 2021

radical mentioned this issue Jun 1, 2021

[wasm][aot] Force python to default to UTF8 #53557

Merged

ghost added the in-pr There is an active PR which will close this issue when it is merged label Jun 1, 2021

radical added a commit to radical/runtime that referenced this issue Jun 1, 2021

[wasm][aot] Force python to default to UTF8

c731714

Fixes dotnet#53367

radical closed this as completed in #53557 Jun 2, 2021

ghost removed the in-pr There is an active PR which will close this issue when it is merged label Jun 2, 2021

qinhuaihe mentioned this issue Jun 3, 2021

publish blazor wasm project with aot enabled failed #53654

Closed

radical mentioned this issue Jul 2, 2021

Set PYTHONUTF8=1 for python pack dotnet/emsdk#35

Merged

ghost locked as resolved and limited conversation to collaborators Jul 19, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Error if user in no python utf8 encoding ENV to run blazor AOT #53367

Error if user in no python utf8 encoding ENV to run blazor AOT #53367

newbe36524 commented May 26, 2021 •

edited

Loading

SIkebe commented May 26, 2021

dotnet-issue-labeler bot commented May 27, 2021

ghost commented Jun 1, 2021

Describe the bug

To Reproduce

Further technical details

qinhuaihe commented Jun 3, 2021

radical commented Jun 3, 2021

newbe36524 commented Jun 3, 2021

Error if user in no python utf8 encoding ENV to run blazor AOT #53367

Error if user in no python utf8 encoding ENV to run blazor AOT #53367

Comments

newbe36524 commented May 26, 2021 • edited Loading

Solution , click here #53367 (comment)

Describe the bug

To Reproduce

Further technical details

SIkebe commented May 26, 2021

dotnet-issue-labeler bot commented May 27, 2021

ghost commented Jun 1, 2021

Describe the bug

To Reproduce

Further technical details

qinhuaihe commented Jun 3, 2021

radical commented Jun 3, 2021

newbe36524 commented Jun 3, 2021

newbe36524 commented May 26, 2021 •

edited

Loading