Android builds with New Arch fail on Windows build host

[birdofpreyru]

Description

The following line is wrong, because glob expects patterns with / separators, even on Windows; but when the codegen is executed on Windows the file will contain \ separators, causing that query not matching anything:

react-native/packages/react-native-codegen/src/cli/combine/combine-js-to-schema-cli.js

Line 25 in ca0d565

.sync(`${file}/**/*.{js,ts,tsx}`, {

Changing that line into:

.sync(`${file.replace(/\\/g, '/')}/**/*.{js,ts,tsx}`, {

fixes the issue (well, I verified that codegen then matches expected files there, but it still does not generate expected outputs, so I am still looking for other issues down the line).

React Native Version

0.71.4

Output of npx react-native info

N/A

Steps to reproduce

N/A

Snack, code example, screenshot, or link to a repository

N/A

[cortinico]

Thanks for reporting this. Yup we should fix this one. Are you up for sending a PR against codegen?

[birdofpreyru]

Yeap, I'll probably do the next week. Still have to figure out what else doesn't work right there.

[birdofpreyru]

Hey @cortinico , I created a PR in Codegen above, fixing schema.json generation on Windows, in a slightly better way.

I tried to further look into why my build still fail, could not figure it out so far. As you see in the screenshot below, it now fails in Ninja build step, saying "No such file or directory", but as you see in the lower half of the screenshot, that folder is actually created by Ninja, although empty. I tried to look around for some additional info in the logs / ninja.build file, so far could not understand why it happens. Saw in Google, it might be a consequence of long pathnames and little CMAKE_OBJECT_PATH_MAX value, and indeed I had a warning about it in logs, so I tried to bump the value up to 4096 – it cleared warnings about paths being dangerously long, but did not change error reported by Ninja.

Thus, I decided to make a pause, create the PR with the first bit of the fix, and ask you for advice, if you have ideas what can be the trouble with Ninja, and what / where should I look to figure it out?

[cortinico]

Thus, I decided to make a pause, create the PR with the first bit of the fix, and ask you for advice, if you have ideas what can be the trouble with Ninja, and what / where should I look to figure it out?

Let's merge the first PR fix and get back to it 👍 I don't normally develop on Windows but I do have a windows machine and can help debug. Also @mganandraj could help as well here

[birdofpreyru]

@cortinico I've changed the PR. It turned out that windowsPathsNoEscape option, I used in the previous variant, requires glob@9+, and the current codegen dependency is glob@7.1.1. Updating glob is a bit too much for me, because flow-typed does not have definitions for v9+, and I don't want create it myself at this time. So, I fell back to the separators replacement in file string. Tested it works, and the PR currently fails just one test in Circle, and that failure does not seem related to my change.

I don't normally develop on Windows but I do have a windows machine and can help debug.

Same here 😅 I just happened to enter a rabbit hole with developing @dr.pogodin/react-native-static-server library, and although not many people use it yet, some already asked about Windows support, and that's how I bumped into this issue here 🤷‍♂️

[mganandraj]

Hi @birdofpreyru
I guess your windows machine don't have long file paths enabled. Could you try doing that ?

Ref: https://learn.microsoft.com/en-us/windows/win32/fileio/maximum-file-path-limitation?tabs=registry#enable-long-paths-in-windows-10-version-1607-and-later

[mganandraj]

And the schema generation from a typescript file work fine on my Windows machine without fixing the slashes the directory input .. Could you share some more details ? such as which shell your are running from, and the full command line ?

[birdofpreyru]

Hi @mganandraj 👋

I guess your windows machine don't have long file paths enabled. Could you try doing that ?

Tried, it does not help.

which shell your are running from

PowerShell 7.3.3

Could you share some more details / and the full command line

So, I am working on this RN library, set up using create-react-native-library for scaffolding. All standard stuff: npm install in the codebase root, and in the /example sub-folder, then doing npm run android inside /example. That works for me on Windows if the new arch is disabled in gradle.properties, fails otherwise without fixes. I checked explicitly adding debug output around

react-native/packages/react-native-codegen/src/cli/combine/combine-js-to-schema-cli.js

Lines 24 to 28 in e188585

	const dirFiles = glob
	.sync(`${file}/**/*.{js,ts,tsx}`, {
	nodir: true,
	})
	.filter(element => filterJSFile(element, platform));

that if file contains \ the glob.sync does not match anything, generates empty schema, while with \ replaced by / it matches files. It is consistent with what glob README tells about slashes — by default it considers \ as escape characters, not path separators, even on Windows.

[mganandraj]

Hi @birdofpreyru
I checked out the repo and built the gradle tasks separately (generateCodegenSchemaFromJavaScript and generateCodegenArtifacts) .. The schema file and artefacts are getting generated .. Console.log at the file enumeration shows the following in my powershell console,
../src/Barrier.ts
../src/constants.ts
../src/Emitter.ts
../src/index.tsx
../src/NativeReactNativeStaticServer.ts
../src/ReactNativeStaticServer.ts
../src/Semaphore.ts

But the schema file and artefacts are generated at react-native-static-server\android\build\generated\source\codegen, and not under "examples" ! Looks like the repo has a react-native project nested within another which is confusing the codegen scripts ..

[birdofpreyru]

@mganandraj well... I don't know... maybe you have something configured for PowerShell to use Linux-style path separators, if there is such option? I just got it installed as it comes, and when I do .\gradlew.bat generateCodegenSchemaFromJavaScript inside \example\android folder of my repo, I got printed from my debug statements in the current codegen:

• console.log(fileList) added to the line 25 prints: [ '..\\src' ]
• console.log(allFiles) added to the line 38 prints: []

Looks like the repo has a react-native project nested within another which is confusing the codegen scripts

This is the standard project layout by https://www.npmjs.com/package/create-react-native-library : the RN library is in the root of the codebase, and an example app using it is inside /example, with example's Metro config set up to pick up the library from the source in parent folder, rather than from /example/node_modules; and this setup works fine when I do Android-target build on Ubuntu; iOS-target build on MacOS; and Windows-target build on Windows; only getting issues with Android-target build on Windows.

[mganandraj]

Well ... If the schema generation works fine with windows-target build on Windows, then it is probably not an issue with the file enumeration in JS code or when dealing windows path, right ? Probably something differs in the gradle tasks .. One related change made last month was to use relative paths when building on Windows host ..

Could you try changing line#81 to "jsRootDir.asFile.get().absolutePath"

[birdofpreyru]

Well ... If the schema generation works fine with windows-target build on Windows

Well... it does not. I just worked it around by manually executing in the codebase root:

.\node_modules\.bin\react-native-windows-codegen
   --libraryName RNReactNativeStaticServerSpec
   --file .\src\NativeReactNativeStaticServer.ts
   --namespace winrt::ReactNativeStaticServer
   --modulesWindows true

then moving and committing generated NativeReactNativeStaticServerSpec.g.h as a part of library code. With this done, then Example app builds and run from MSVS. Without it, it does not generate that header (and schema i guess), and the Example build fails because of it.

[cortinico]

(cc @kelset if we need to pass this to someone at Microsoft)

[kelset]

cc @ZihanChen-MSFT maybe you know how to help here?

[mganandraj]

Sorry .. I got extremely busy last week .. Adding @rasaha91 and @shivenmian who can help as well .

[birdofpreyru]

For the record, it looks like PR #39190 aims to fix the CMake part of the issue.

[cortinico]

@birdofpreyru as #39190 is merged, can we close this one?

[birdofpreyru]

I usually like to keep my tickets open until the fix is released with a regular version of a product, and the issue is verified to be fixed by it. Especially in this case, I have no idea yet whether that and other PRs will completely fix the problem, or not.

[cortinico]

I usually like to keep my tickets open until the fix is released with a regular version of a product, and the issue is verified to be fixed by it. Especially in this case, I have no idea yet whether that and other PRs will completely fix the problem, or not.

Great I've assigned this to you so we know you're on top of it. I general we tend to close issues more proactively as we have so many, and we close issues once a fix lands on main and eventually re-open. I'm fine keeping this open though

[cortinico]

I usually like to keep my tickets open until the fix is released with a regular version of a product, and the issue is verified to be fixed by it

@birdofpreyru I'm closing this one as we're grooming New Architecture issues. I know that the fix hasn't been shipped yet (will be in 0.73), we can reopen this issue if the problem pops back.

[birdofpreyru]

Hey @cortinico , at last I arrived to do a rapid test, trying to build the example app of my lib for Android on Windows with the new RN arch, and it still does not work for me :(

According to my understanding of this log piece, it seems the JNI project generated by RN Codegen easily hits the CMake restriction on the maximum path length on Windows. Perhaps setting in the generated CMakeLists.txt that CMAKE_OBJECT_PATH_MAX variable, mentioned in the log, to a larger value will help. Though, as I don't really need myself to build for Android on Windows, I probably won't look deeper into it in the nearest future. Just leaving this message for a record, and probably you may re-open the ticket.