API to get available view properties

[brunolemos]

Feature request

Allow get the current visible text or maybe any props from a matched element.

Use case

I'm using detox to run automated benchmarks for my repo react-native-css-in-js-benchmarks. I'm able to make detox tap on things to run the benchmark, but I see no way to pass the benchmark results from the app to detox.

So a way to get the printed result OR a method to pass data to detox would be great to expand detox use cases. Any thoughts about this?

---

Similar stack overflow question that I found: https://stackoverflow.com/questions/47351745/how-to-get-the-text-from-a-matched-view-using-detox

[rotemmiz]

This can be a cool feature, and as you said, will greatly expand the use cases covered by Detox.
At the moment this is not on our roadmap, but we will help anyone who wants to take this and submit a PR.

[brunolemos]

@rotemmiz Thanks for the answer. Could you point here some directions for any interested contributor? E.g. which files they would probably touch or existing methods they would probably use.

[rotemmiz]

• On iOS I'd start with EarlGrey's FAQ: https://github.com/google/EarlGrey/blob/3d8489c6bf1f4f5619ad3dac7154fc5b07b78176/docs/faq.md#is-there-a-way-to-return-a-specific-element

• On Android, should be something like: https://stackoverflow.com/a/23467629

Both will have to be implemented natively in Detox as Matchers, in here (iOS) and here (Android)

Once these are are done, the code generator should be able to pick this up and and generate the JS API.

[LeoNatan]

Matching “props” to actual native properties is the hardest here. Starting with “simple” stuff like text should be easier. I say “simple” because that too can blow up in complexity—labels, text fields, text views, custom views, etc.

[brunolemos]

Yep, props are not really necessary initially, just a plus for the future.
The visible text could use a similar implementation from the by.text matcher.

[saitejareddy07]

do you know guys know when these can be done

[brunolemos]

@saitejareddy07 we need someone to volunteer to work on it.
I'm not a native developer unfortunately and wix team seems busy with other tasks.

[saitejareddy07]

I agree with you wix team has to consider this issues, Detox is completely uni-directional, it's not bi-directional

[edwardsfriedman]

@LeoNatan @rotemmiz Any movement or news on this? Love detox! Swapping over from appium for my project, regardless. But this still seems like it'd be a handy feature.

[LeoNatan]

Hello,

Unfortunately we have not looked at implementing this yet. PRs are welcome.

[shockdesign]

This would be perfect as its something that would help with tests being less flaky.*

• When not being able to control test data.

[yogeshthanvi]

Hi guys anyone have worked on this

[Siva123Prasad]

Looks like no one as of now.

[MiroslavPetrik]

This would be a great feature - reading text from the screen to assert that correct values are displayed or just read the state...

[Kamahl19]

This is a workaround we use to get text from the element

export async function readTextValue(testID) {
  try {
    await expect(element(by.id(testID))).toHaveText("__read_element_error_");
  } catch (error) {
    const start = `AX.id='${testID}';`;
    const end = "; AX.frame";
    const errorMessage = error.message.toString();
    const [, restMessage] = errorMessage.split(start);
    const [label] = restMessage.split(end);
    const [, value] = label.split("=");

    return value.slice(1, value.length - 1);
  }
}

[cs01]

@Kamahl19 nice hack, worked for me!

[MiroslavPetrik]

@Kamahl19 Thats a really nice hack! Who would say that the error message contains the actual value, lol.

Is the testID always followed by the AX.frame property?

[Kamahl19]

@MiroslavPetrik I am not sure 100% AX.frame string is always there. This is just a hack you can use and customise for your needs in your project. I hope someone will add a proper native API for this soon.

[visopsys]

I do not know why the authors do not want to support get attributes from views in testing since it is not really hard. There might be some design philosophy that they specifically avoid doing this that I am not aware of.

In case you are really in need of getting text in test, this is an example of how I do it. You can get not just text but also potentially get width/height/X,Y coordinate/ parent views or anything in test.

86f478f

Basically you add return values in a bunch of async functions. My commit only works on Android since that's all what I need. I hope implementing on iOS should not be too difficult 😄

[ronilitman]

Any news with that?

[ronilitman]

@Kamahl19 This doesn't work for me on Android, didn't try on iOS yet. Does it work for you in the current detox version?

[yogeshthanvi]

Hi @ronilitman We are using below code to fetch text
const readVisibleText = testID => {
return new Promise(async resolve => {
if (device.getPlatform() === 'ios') {
try {
await expect(element(by.id(testID))).toHaveText('_read_element_error');
} catch (error) {
console.log(platform is : ${device.getPlatform()});
const start = AX.id='${testID}';;
const end = '; AX.frame';
const errorMessage = error.message.toString();
const [, restMessage] = errorMessage.split(start);
const [label] = restMessage.split(end);
const [, value] = label.split('=');
resolve(value.slice(1, value.length - 1));
}
} else {
try {
await expect(element(by.id(testID))).toHaveText('_read_element_error');
} catch (error) {
console.log(platform is : ${device.getPlatform()});
const start = Got:;
const end = '}"';
const errorMessage = error.message.toString();
const [, restMessage] = errorMessage.split(start);
const [label] = restMessage.split(end);
const value = label.split(',');
var combineText = value.find(i => i.includes('text=')).trim();
const [, elementText] = combineText.split('=');
console.log(combineText);
resolve(elementText);
}
}
});
};

export default readVisibleText;

[maxime-helen]

I took the initiative to edit above. This works with 12.7.0, at least for ios.

const readVisibleText = async (testID) => {
  try {
    await expect(element(by.id(testID))).toHaveText('_unfoundable_text');
    throw 'We never should get here unless target element has unfoundable text';
  } catch (error) {
    if (device.getPlatform() === 'ios') {
      const start = `accessibilityLabel was "`;
      const end = '" on ';
      const errorMessage = error.message.toString();
      const [, restMessage] = errorMessage.split(start);
      const [label] = restMessage.split(end);
      return label;
    } else {
      // Android to be tested
      const start = 'Got:';
      const end = '}"';
      const errorMessage = error.message.toString();
      const [, restMessage] = errorMessage.split(start);
      const [label] = restMessage.split(end);
      const value = label.split(',');
      var combineText = value.find(i => i.includes('text=')).trim();
      const [, elementText] = combineText.split('=');
      return elementText;
    }
  }
};

[LeoNatan]

As mentioned elsewhere, we are working on an overhaul of how Detox works. This will become impossible once we move to that system (no direct access to the controls).

[plaa]

@LeoNatan Can you elaborate on the overhaul (is it documented somewhere)? Will it become impossible to fetch a piece of text from the screen?

We have many use cases where we need to read a number from the screen and verify that it's within a specific range. If this will become impossible in Detox, we'll need to move to a different E2E testing framework. We'd prefer to do the change before we put a lot of time in writing tests using Detox.

[LeoNatan]

There is no documentation. We will be using Apple's UI testing framework internally, which runs in a different process than the tested app.

We will only have access to the following view properties:
https://developer.apple.com/documentation/xctest/xcuielementattributes?language=objc

Those properties can be exposed to the user facing JS API somehow. However, we will not have direct access to the view object, and thus we will not be able to fetch any React information.

If you are inspecting the text information, that is accessible using the label property.

[cs01]

This will become impossible once we move to that system (no direct access to the controls).

The workaround presented in this issue dumps the UI tree and manually parses out the label. It looks like after the overhaul it will become easier rather than impossible since there is an official API that exposes the label property.

[plaa]

@LeoNatan How will Android be supported?

[LeoNatan]

@cs01 Yes, as long as it is one of the supported properties posted above, it will be possible to transfer them from native to JS. However, this requires that React Native support platform standard conventions, which sometimes they don't. For instance, a custom view cannot implement accessibilityValue and thus the value property will not have the correct value. There is no way in RN to use a subclass of UIControl, which means that custom controls, such as custom switches and toggles, will not be seen as such in the accessibility system, and so on.

This was my initial reluctance to support a system as requested in this issue. Coupled with many people's lack of understanding that Detox runs in native exclusively, and that native views are not React views, and their props don't magically show up in native—I fear will introduce a lot of confusion.

@plaa No plans to make extensive changes currently.

[cs01]

In our tests, we very often need to get the value of an element and assert on it. We have adopted the workaround described in this issue for now, and it has been working nicely for us.

This issue title is regarding getting props or visible text. Agreed that props is not possible since there are transformations done between the React Native code and Native code, but the visible text is available at the native layer. The UI tree used in this workaround is the Native tree, not the original React Component tree, and it seems to be useful for many others besides us.

I hope there is some way we could get an official Detox API for this, and at the same time set expectations that only the Native values, not the original React code or props, will be accessible by Detox.

[LeoNatan]

Right, but unfortunately, title/label are not always enough. As I mentioned above, control values are not accessible for some RN cases, where people would expect them to be. But since we are designing a new internal API around this, we will think how to perhaps expose these properties. Perhaps it can be a second step, such as getViewProperties(element(by.id('asd'))); which would return a json with all the linked properties for the user to do with as needed.

[LeoNatan]

OK, let's keep open.

[nschurmann]

What's the status on this? I'm interested on this.

[TammyTorres]

@LeoNatan There is some update on this? Would be very useful for my team

[LeoNatan]

We may have an update on this soon.

[LeoNatan]

Good news, this was recently implemented, and will soon find its way into a release.

For now, see the API here:
https://github.com/wix/Detox/blob/DetoxNext/docs/APIRef.ActionsOnElement.md#getAttributes--ios-only

Thanks

[plaa]

@LeoNatan Any solution for Android? We're currently running our tests only on Android, so this doesn't help us at all. 😞

The attributes are platform-dependent, but I'd suggest harmonizing some field e.g. text for the most used case of fetching text.

[LeoNatan]

Android will be implemented in the future. No timeline currently. I’m sure we’ll keep the attribute keys as similar as possible.

[plaa]

I created #2083 so that it's possible for people to track when support on Android is implemented. Until then, will have to continue using detox-getprops.

[oguennec]

@LeoNatan on iOS with detox 16.8.2, I am getting:
TypeError: element(...).getAttributes is not a function

when running according to your doc link above:
const attributes = await element(by.id('detoxTest')).getAttributes();

while:
await expect(element(by.id('detoxTest'))).toBeVisible(); works.

[LeoNatan]

@oguennec It’s not deployed yet. Check out the pull request to follow the progress.