Introduce Gemfile, ruby-version

[barbieri]

Summary

Implement par of the discussion react-native-community/discussions-and-proposals#411, except the .nvmrc part, this includes:

• Setting .ruby-version in the main project and also template/
• Fixing the CocoaPods version with a project-level Gemfile and also template/Gemfile
• Using all pod executions from bundle exec pod, using the determined version
• Script to manage and update the ruby version

Changelog

[iOS] [Added] - Gemfile with CocoaPods 1.11 and ruby-version (2.7.4)

Test Plan

Build for iOS and run all CircleCI tests to see if nothing changed

[analysis-bot]

Platform	Engine	Arch	Size (bytes)	Diff
android	hermes	arm64-v8a	7,714,506	+0
android	hermes	armeabi-v7a	7,243,229	+0
android	hermes	x86	8,134,226	+0
android	hermes	x86_64	8,099,580	+0
android	jsc	arm64-v8a	9,633,792	+0
android	jsc	armeabi-v7a	8,550,029	+0
android	jsc	x86	9,647,800	+0
android	jsc	x86_64	10,256,835	+0

Base commit: 34b7d22

[fkgozali]

Looks like the build failures may be related?

#!/bin/bash -eo pipefail
node ./scripts/publish-npm.js --dry-run
Setting Ruby version to: 2.7.0
sed: can't read : No such file or directory
Failed to update Ruby version
Failed to bump version number

Exited with code exit status 1
CircleCI received exit code 1

[barbieri]

I'll take a look tomorrow, maybe change to set -xe so we can see where it failed... It's weird because

sed -i '' -e "s/^\(ruby '\)[^']*\('.*\)$/\1$VERSION\2/" Gemfile
sed -i '' -e "s/^\(ruby '\)[^']*\('.*\)$/\1$VERSION\2/" template/Gemfile

and both files were added in the commit, but I need to see if stuff was copied, maybe I forgot to add those files.

[barbieri]

@fkgozali do you have more knowledge on this?

+ cd /root/react-native/scripts/..
...
+ sed -i '' -e 's/^\(ruby '\''\)[^'\'']*\('\''.*\)$/\12.7.0\2/' Gemfile

is the /root/react-native the checkout or something else? why wouldn't Gemfile be there?

[barbieri]

also, is it linux or macOS? Because sed -i behaves different in both cases, so maybe that's the reason

[fkgozali]

@fkgozali do you have more knowledge on this?

+ cd /root/react-native/scripts/..
...
+ sed -i '' -e 's/^\(ruby '\''\)[^'\'']*\('\''.*\)$/\12.7.0\2/' Gemfile

is the /root/react-native the checkout or something else? why wouldn't Gemfile be there?

Hmm it should be the checkout. I'm actually not sure here, you might need to add a bit of more logs (echo?) to debug further.

The iOS tests are always running on MacOS, e.g. https://app.circleci.com/pipelines/github/facebook/react-native/10546/workflows/27b085f4-3142-4453-9125-cbbaea93d3e0/jobs/219844

[barbieri]

added some more safety, just updates from macOS... I think that's the sed -i differences between Linux and macOS, the Linux doesn't recognize the empty string argument as the extension (ie: no backup) and instead will consider the empty string as the file to be updated :-/

Since the Ruby part is only interesting for macOS, I'm ignoring updates from other OS.

[barbieri]

@fkgozali seems that one is fixed, but still some errors. Could you check what are they about?

test_ios_unit_jsc - yarn test-ios and test_ios_unit_hermes both fail in a similar [CP-User] Generate Specs phase, this could be related, but hard to say without actual logs:

PhaseScriptExecution [CP-User]\ Generate\ Specs /Users/distiller/Library/Developer/Xcode/DerivedData/RNTesterPods-cbjldxeeenzmdtbkoznuxxultmjt/Build/Intermediates.noindex/Pods.build/Debug-iphonesimulator/FBReactNativeSpec.build/Script-D50C3B18C67D0D1A8D380643D5AEF835.sh (in target 'FBReactNativeSpec' from project 'Pods')

Others looks unrelated:

• test_docker_android is failing echo y | npx envinfo@latest
• analyze_pr fails using apt repository https://deb.nodesource.com/node_14.x

[fkgozali]

test_ios_unit_jsc - yarn test-ios and test_ios_unit_hermes both fail in a similar [CP-User] Generate Specs phase, this could be related, but hard to say without actual logs:

These are unrelated, it was due to Xcode 13 upgrade, but we're not sure why yet.

[barbieri]

test_ios_unit_jsc - yarn test-ios and test_ios_unit_hermes both fail in a similar [CP-User] Generate Specs phase, this could be related, but hard to say without actual logs:

These are unrelated, it was due to Xcode 13 upgrade, but we're not sure why yet.

some google says it's related to build order, I'll take a look if I can fix that in another PR.

As for the current approach regarding ruby-version + Gemfile, let me know what you guys think about that.

[fkgozali]

Re: build order, sounds good. FYI it started failing in this commit: https://app.circleci.com/pipelines/github/facebook/react-native/10525/workflows/b68b10a3-325a-4892-8252-baed9017c613

For the rest of the solution, we'll take a look tomorrow.

[fkgozali]

also cc @mikehardy for inputs

[fkgozali]

The other error should be fixed by #32309

[liamjones]

Might I suggest also adding a bundler config to tell bundler to install the dependencies locally in the project? Again, to keep things independent of global installed deps:

.bundle/config:

---
BUNDLE_PATH: "vendor/bundle"

And then add /vendor to the .gitignore.

[mikehardy]

I'm not good enough with ruby to offer much, but I do worry if it doesn't work on Linux for android only template init build tests (I do this sometimes) or if there's no way to override (I use ruby 3.0.2 personally). Those are feature specification hopes not code comments though. I worry a little about future issue traffic from the ruby ignorant (like myself) about all things related to this, I fear it opens a support surface area for ruby distribution, upgrades, gem files etc), and I'm curious if it will be balanced by a decrease in issue count around incorrect ruby versions, or incorrect pod versions.

[barbieri]

I'm not good enough with ruby to offer much, but I do worry if it doesn't work on Linux for android only template init build tests (I do this sometimes) or if there's no way to override (I use ruby 3.0.2 personally). Those are feature specification hopes not code comments though. I worry a little about future issue traffic from the ruby ignorant (like myself) about all things related to this, I fear it opens a support surface area for ruby distribution, upgrades, gem files etc), and I'm curious if it will be balanced by a decrease in issue count around incorrect ruby versions, or incorrect pod versions.

It will work on Android, but in android there is no need to pod install. Currently the usage of Gems is only to get CocoaPods. In the future we could isolate parts of that with platform blocks/groups, in that case I'd have to fix the sed call to be different on Linux (easy, but the current "fix" is simpler and more reliable, as no ruby/gems is used outside of macOS).

As for use another version of Ruby, you definitely can, in particularly if you create a new repo (react native init), you're free to change the versions there, but you may run into problems since it's not as tested.

We're sticking to the "latest stable", and it's the same as CircleCI (and likely other CI) uses.

[mikehardy]

Those are certainly reasonable responses. My comments were all non-quantitative, they were qualitative, so any reasonable defense can be thought of as sufficient I think. Works for me and we'll see I think :-)

[barbieri]

@mikehardy the new version uses the sed_i function to allow it to run on Linux.

@liamjones it also uses .bundle/config

However, I noticed something: the _-prefixed files are not being replaced automatically as I thought, instead we'd need to convert them by changing https://github.com/react-native-community/cli/blob/master/packages/cli/src/tools/generator/copyProjectTemplateAndReplace.ts#L135-L150 so I need to send a patch there, if those are to be used and also fix scripts/run-ci-e2e-tests.js (done).

[mikehardy]

@barbieri https://www.youtube.com/watch?v=AbSehcT19u0 😆

[barbieri]

@barbieri https://www.youtube.com/watch?v=AbSehcT19u0 😆

exactly! :-D

[fkgozali]

@barbieri @mikehardy thanks for the inputs. To be honest, I have little context on Ruby/Gemfile setup, but I did find myself struggling to make sure my env was correct, usually around random Ruby failures. So from that angle alone, IMO "any signals to indicate that something is misconfigured in your system" is very valuable. I understand the potential risks of us getting asked non-React Native questions like Ruby stuffs, but perhaps it's gonna be just in the short term (we'll see).

So I'll import this PR and get some more inputs from our team at FB, and if there's no concern, we'll merge it. I see this as an incremental improvement though, so we can always tweak things as needed and make things more robust.

[fkgozali]

@barbieri 34b7d22 just landed, which should fix the iOS tests. Could you rebase this PR to pick it up so we can be sure your PR doesn't regress anything else? After that I'll import

[friederbluemle]

Nice work @barbieri 👍

I do have one question though (Please correct me if I'm wrong): These changes are primarily for working with the react-native repo itself (e.g. running e2e tests, CI, etc), correct?
If yes, can we - by any chance - keep the basic app template clean and minimal without the overhead of additional Ruby/Gemfile/Bundler files (i.e. undo the changes inside of the template/ directory)?

There already is a clearly written section about the requirement to install CocoaPods in the official React Native environment setup guide (cc @thymikee).

I'm quoting one section:

You can use a Ruby Version manager, however we recommend that you use the standard Ruby available on macOS unless you know what you're doing.

To me it sounds like everything should still work for someone who set up their environment following the guide. I don't have an M1 mac available to test right now (in react-native-community/discussions-and-proposals#411 you mentioned "ffi on arm64/m1") - Are there any issues with it and the previous template?

[barbieri]

Btw, there's an internal test failure (I'm looking at it) related to the _bundle dir in the new app template. In the meantime, it would be useful to validate that this flow works with the new app template as well. You can try the .tar.gz artifact with the npx react-native init CLI to create a brand new app: https://app.circleci.com/pipelines/github/facebook/react-native/10566/workflows/99007ae2-648e-4389-a218-4052da0eda03/jobs/220064/artifacts

I'll take a look what's the exact problem, but I know I need to modify the cli project to convert the new _-prefixed files to ..

[barbieri]

Nice work @barbieri 👍

I do have one question though (Please correct me if I'm wrong): These changes are primarily for working with the react-native repo itself (e.g. running e2e tests, CI, etc), correct? If yes, can we - by any chance - keep the basic app template clean and minimal without the overhead of additional Ruby/Gemfile/Bundler files (i.e. undo the changes inside of the template/ directory)?

Yes, they work (at least with the CI and @fkgozali internal tests as well. The idea of template/ is to make it easier, not more complex, by bringing some well known versions, if you reduce the variations you're likely to reduce the number of issues. For example CocoaPods 1.11 works much better than 1.10 for newer Apple hardware (M1). Even CircleCI recommends that: https://circleci.com/docs/2.0/testing-ios/#images-using-xcode-117-and-later

As a result of the macOS system Ruby (2.6.3) becoming increasingly incompatible with various gems (especially those which require native extensions), Xcode 11.7 and later images default to Ruby 2.7 via chruby. Defaulting to Ruby 2.7 allows for greater compatibility and reliability with gems moving forward. Common gems, such as Fastlane, run without any issues in Ruby 2.7.

There already is a clearly written section about the requirement to install CocoaPods in the official React Native environment setup guide (cc @thymikee).

That doesn't state the recommended version, more than that, it will install global in the system, which may cause issues with people working in multiple projects (common if you have to do maintenance in one project while creating a new one).

I'm quoting one section:

You can use a Ruby Version manager, however we recommend that you use the standard Ruby available on macOS unless you know what you're doing.

To me it sounds like everything should still work for someone who set up their environment following the guide.

It seems it will still work, the Gemfile and .ruby-version are not strictly required, you cam gem install yourself, but if you call via bundle exec it will first check the version. Of course you're supposed to use the "tested version", but you can just ignore, remove the file or do whatever you want. Same for .ruby-version, that will help to get the proper environment setup, but if you do not have rbenv or rvm, the environment won't be changed.

I don't have an M1 mac available to test right now (in react-native-community/discussions-and-proposals#411 you mentioned "ffi on arm64/m1") - Are there any issues with it and the previous template?

To make sure I cleaned everything and did a new setup: there is no real problem, seems ruby 2.6.3 DOES WORK with ffi_c.bundle IF AND ONLY IF you never messed up with gems and rosetta2.

What happened (at least to me and some colleagues) is that many tutorials to work around M1 limitations did recommend to execute the processes in Rosetta environment. If you did that, the files may be of an incorrect architecture, ex:

# correct
$ find /Library/Ruby/Gems -name 'ffi_c.bundle' | xargs file                                 
/Library/Ruby/Gems/2.6.0/extensions/universal-darwin-20/2.6.0/ffi-1.15.4/ffi_c.bundle: Mach-O 64-bit bundle arm64
/Library/Ruby/Gems/2.6.0/gems/ffi-1.15.4/ext/ffi_c/ffi_c.bundle:                       Mach-O 64-bit bundle arm64
/Library/Ruby/Gems/2.6.0/gems/ffi-1.15.4/lib/ffi_c.bundle:                             Mach-O 64-bit bundle arm64

# incorrect, ex: arch -x86_64 sudo gem install cocoapods
$ find /Library/Ruby/Gems -name 'ffi_c.bundle' | xargs file
/Library/Ruby/Gems/2.6.0/extensions/universal-darwin-20/2.6.0/ffi-1.15.4/ffi_c.bundle: Mach-O 64-bit bundle x86_64
/Library/Ruby/Gems/2.6.0/gems/ffi-1.15.4/ext/ffi_c/ffi_c.bundle:                       Mach-O 64-bit bundle x86_64
/Library/Ruby/Gems/2.6.0/gems/ffi-1.15.4/lib/ffi_c.bundle:                             Mach-O 64-bit bundle x86_64

If any of the ffi_c.bundle is x86_64, then people may have issues with that. The simplest "fix" is to uninstall everything and install again without Rosetta2: sudo gem uninstall -Iax

EDIT-1: listed both arm64 and x86_64 output to avoid confusion

[fkgozali]

I'll take a look what's the exact problem, but I know I need to modify the cli project to convert the new _-prefixed files to ..

The problem for FB internal was the command used to move _ to .. It used find -H * ... which works fine for files, but failed on dirs. I had to modify it to find -H . ...

[MoOx]

Any reason vendor/bundle hasn't been added into template _gitignore ?

[barbieri]

Any reason vendor/bundle hasn't been added into template _gitignore ?

hum... I forgot, nice catch :-)

[kelset]

@MoOx @barbieri if either of you makes a PR to address that, can you tag me in it so that we don't forget to cherry pick for 0.67?

[MoOx]

@kelset what branch should the PR be pointed at?

[kelset]

main, and then we cherry pick to 0.67 branch

[mikehardy]

Now that this is out and live, it immediately causes me a problem:

/Users/mike/.rvm/gems/ruby-3.1.0/gems/bundler-2.3.5/lib/bundler/definition.rb:426:in `validate_ruby!': Your Ruby version is 3.1.0, but your Gemfile specified 2.7.4 (Bundler::RubyVersionMismatch)
        from /Users/mike/.rvm/gems/ruby-3.1.0/gems/bundler-2.3.5/lib/bundler/definition.rb:401:in `validate_runtime!'

I'm running current stable version of ruby, and I try to force everything to use that, so I do not have, and do not want ruby-2.7.4 on my system. I understand the desire for react-native to have a known version to work with, but having it force the issue at a step where I cannot intercept it (during npx react-native init) in order to modify the Gemfile or anything is not something I want.

For now I have to install the old ruby just to run scripted compile tests that start with react-native init I guess

But is there a way to make this optional? Or make it accept any version above the minimum, like >=2.7.4 ?

[mikehardy]

I can use --skip-install on the npx react-native init line, then sed the Gemfile to have ruby '>= 2.7.4' then do yarn && pushd ios && pod install && popd and it works.

Having the >= for ruby version by default seems like it would allow for perhaps some problems but would also work better for the reasonable case where 2.7.4 had long ago been abandoned by the user

[barbieri]

I can use --skip-install on the npx react-native init line, then sed the Gemfile to have ruby '>= 2.7.4' then do yarn && pushd ios && pod install && popd and it works.

Having the >= for ruby version by default seems like it would allow for perhaps some problems but would also work better for the reasonable case where 2.7.4 had long ago been abandoned by the user

maybe doesn't help due the lockfile, but since you're running it, have a try if that changes (Gemfile.lock).

[ravirajn22]

I am facing trouble due to this change, below are my doubts.

1. When I run bundle exec pod install, I get Your Ruby version is 2.6.8, but your Gemfile specified 2.7.4. So should I change system level Ruby version?
2. https://reactnative.dev/docs/environment-setup the docs still says to install global cocoapods. No doc related to these changes.
3. Is these changes ported to npx react-native, ie react-native-cli?
4. Why do we include the Gemfile.lock file in the template? Should not the end user just generate it?
5. Should we bundle install before bundle exec pod install?

[liamjones]

@ravirajn22 if you install rbenv or rvm that can use the .ruby-version file to ensure the required version of ruby is available for the project without affecting your system version

[Bardiamist]

I want to try the New Architecture
But my Ruby version was lower than 2.7.5 so I installed rbenv, configured... and now
ruby -v and even bundle exec ruby -v returns:

ruby 2.7.5p203 (2021-11-24 revision f69aeb8314) [arm64-darwin21]

But RCT_NEW_ARCH_ENABLED=1 bundle exec pod install returns:

Your Ruby version is 3.1.2, but your Gemfile specified 2.7.5

Why? How to use ruby 2.7.5 here?

[kelset]

@Bardiamist I don't know if it helps, but what happens if you try running RCT_NEW_ARCH_ENABLED=1 pod install? I reckon (based on my experience with rbenv) that what might happen is that it might complain that cocoapods is not installed (basically rbenv makes you install cocoapods per each version of ruby) - but at that point, after installing it from that same folder (so that it will install it for 2.7.5) it should work?

[Bardiamist]

@Bardiamist I don't know if it helps, but what happens if you try running RCT_NEW_ARCH_ENABLED=1 pod install? I reckon (based on my experience with rbenv) that what might happen is that it might complain that cocoapods is not installed (basically rbenv makes you install cocoapods per each version of ruby) - but at that point, after installing it from that same folder (so that it will install it for 2.7.5) it should work?

Ok, thanks, understand more now. Yes RCT_NEW_ARCH_ENABLED=1 pod install works because I has cocoapods installed by sudo gem install cocoapods.

But idea was to run bundle install for to install cocoapods. Look it installed cocoapods by Ruby 2.7.5 and now RCT_NEW_ARCH_ENABLED=1 bundle exec pod install also works.

[mikehardy]

I have to admit I still dislike the whole bundle / Gemfile / .rubyversion thing. I do this every time now:

npm_config_yes=true npx @react-native-community/cli init rnfbdemo --skip-install --version=0.69.4
cd rnfbdemo

# New versions of react-native include annoying Ruby stuff that forces use of old rubies. Obliterate.
if [ -f Gemfile ]; then
  rm -f Gemfile* .ruby*
fi

# Now run our initial dependency install
yarn