Developer: App Test Protocol

[E3V3A]

We need to invent and post a test protocol that can go through a few different situations, that we can detect. The test protocol is meant to be used only for developers and testers to test various features of the app. When app is in beta stage, this test protocol should be extended to test more aspects in a real IMSI-catcher environment.

Basic Tests
__________

• Phone/SIM Information:
  a) Are all items populated / shown?
  b) Are the items correct?
  c) Do the items update immediately when phone changes network or data state?
• Cell Information:
  a) Does the current AIMSICD status show?
  b) Is it correct? ??
  c) Are the neighboring cells shown?
  d) Do they look correct? (Compare with those shown in Service/Engineering Mode.)
  e) Is Ciphering Information shown?
• Monitoring Modes:
  a) Does toggling of Monitoring Mode work as expected?
  b) Does toggling of 2G lock-in mode work as expected?
  c) Does toggling of GPS use work as expected? (Maybe this need to be removed?)
• Antenna Map Viewer:
  a) Does all BTS show?
  b) Are they color coded correctly?
  c) Can you Zoom In/Out?
  d) Does new BTS pins re-populate the map after (c).
  e) Does all details of the BTS's show?
• Database Viewer:
  a) Does all Databases get properly populated? [See list in Major DataBase overhaul and restructuring [$20] #215.]
  b) Are all the DB items (as shown) correct? (I.e. LAC, CID, RSSI, etc.)
• OCID API key:
  a) Does the API key download work?
  b) Does the API key entry work?
  c) Does the API key dispaly work?
  d) Does the Download OCID data work with the private API key you added in (c)?
• AT Command Interface:
  a) Does the AT command interface appear as it should? Compare to this screenshot.
  b) Does the correct /dev/XXX device (for your phone) appear in the drop-down list?
  c) Does sending AT in the text box, result in a OK response?
  d) Does sending ATI in the text box, result in a response with your device info?
  e) Does sending at+cops? in the text box, result in any response? See this list for others.
  f) After closing and re-opening ATCoP interface, does it still work the same?
• Cell Information:
• Preferences:
  a) Does the AIMSICD DataBase backup and restore functions work?
  b) Does the DB table(s) CSV import/export work as expected?
  c) Does the Automatic AIMSICD Service Start-up on phone boot-up toggle, behave as expected?
  d) Does the Persistent Service toggle, behave behave as expected?
  e) Does the FemtoCell Detection behave as it should?
  f) Does the FemtoCell Protection behave as it should?

---

Detection Testing
________________

We can test some detections by manually adding some rows to the DBi_measure and DBi_bts tables, and possibly others. We've also started to implement some code to do this, in the test directory. Here are a few items we would like to test.

• Changing LAC here
• Changing CID
• DBe_import inconsistency
• Silent SMS
  ...

---

NOTE: The items above does not yet reflect what is currently present on the Application. We're slowly working on this, since we're still updating the UI/UX.

[More TBA]

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

[SecUpwN]

Interesting proposal, @E3V3A. Shall that be like a script that tests all functions of the App in IMSI-Catcher environments, FemtoCell situations and such? Please define the situations a little more.

[tobykurien]

Excellent proposal @E3V3A - I've been thinking about tests for the detection algorithms. These tests would need to run independently of the actual cellular radio (just to test that the actual detection algorithm works). This would need an abstraction layer between the detection process and the cellular APIs. This will also help us with supporting various devices.

[SecUpwN]

@tobykurien, maybe we can launch these tests on startup of the App while in "Idle" mode? After all tests have been successfully run, AIMSICD shall turn to the green "OK" status. Just an idea.

[tobykurien]

@SecUpwN oh no, I think we're talking about completely different types of tests. I'm talking about developer testing (unit/functional tests), as per the title of the issue.

[E3V3A]

@tobykurien Yes, good point. I've updated OP and renamed issue to include UI/UX in the title. The RF detection protocol would be a more complicated and completely different thing. And need to be assigned a new issue. TBA.

[He3556]

What about changing values, in one of the two databases? We can change the LAC value of one Cell-ID. We can do this two ways, either:

1. We can to change it in the local OpenCellID DB (DBe_import) or
2. we change it in the locally measured value (from API) in (DBi_meas).

I will try it this way next week.

[E3V3A]

@He3556 Can you post the SQLs needed to do this test? I want all manual testing scenarios, documented here, in anticipation of a complete AppTestProtocol.md page.

[E3V3A]

The easiest way to test this is probably to inject a false LAC but with same CID, into the CELL_TABLE (cellinfo) DB as @He3556 suggested. We can do this with:

# Go to AIMSICD databases directory
cd /data/data/com.SecUpwN.AIMSICD/databases/

# Clear main logcat
logcat -c -b main

# Make a copy of the last row, and insert into the same table
sqlite3 -csv aimsicd.db 'INSERT INTO cellinfo (Lac,CellID,Net,Lat,Lng,Signal,Mcc,Mnc,Accuracy,Speed,Direction,NetworkType,MeasurementTaken,OCID_SUBMITTED,Timestamp) SELECT Lac,CellID,Net,Lat,Lng,Signal,Mcc,Mnc,Accuracy,Speed,Direction,NetworkType,MeasurementTaken,OCID_SUBMITTED,Timestamp FROM cellinfo WHERE _id = (SELECT MAX(_id) FROM cellinfo);'

# Update the last row with fake LAC=11111
sqlite3 -csv aimsicd.db 'UPDATE cellinfo SET Lac=11111 WHERE _id = (SELECT MAX(_id) FROM cellinfo);'

# Find LAC warning in main logcat
logcat -d -b main |grep -iE "LAC"

The result is:

V/AISMICD_DbAdaptor(25860): LAC checked - no change.  CID:682XXXX LAC(DBi):10XX1 LAC(DBe): 10XX1 
I/AISMICD_DbAdaptor(25860): ALERT: Changing LAC on CID: 682XXXX Current LAC(DBi): 10XX1 Database LAC(DBe): 11111

[tobykurien]

FYI, I wrote an Android Unit Test for this here: https://github.com/SecUpwN/Android-IMSI-Catcher-Detector/blob/master/app/src/androidTest/java/com/SecUpwN/AIMSICD/test/Detection1.java

[E3V3A]

Wow, great! How can we use it? Can we call it by using an am like call from command line?
Perhaps by using something like?:

am start -n com.SecUpwN.AIMSICD/.Detection1

[tobykurien]

You can run it either from gradle connectedAndroidTest (which runs all tests) or Android Studio (right-click the class, Run > Detection1). It's also possible to use am instrument to run tests, as described in https://developer.android.com/tools/testing/testing_otheride.html#RunTestsCommand

More here: https://developer.android.com/tools/testing/testing_android.html

[E3V3A]

am instrument [flags] <test_package>/<runner_class>

Parameter	Value	Description
<test_package>	The Android package name of the test package.	The value of the package attribute of the manifest element in the test package's manifest file.
<runner_class>	The class name of the instrumented test runner you are using.	This is usually InstrumentationTestRunner.

# To run all of the test classes in the test package, enter:
am instrument -w com.android.demo.app.tests/android.test.InstrumentationTestRunner

#  To run all of the tests in the class UnitTests, enter:
am instrument -w -e class com.android.demo.app.tests.UnitTests com.android.demo.app.tests/android.test.InstrumentationTestRunner

# To run all of the tests in UnitTests, and the testCamera method in FunctionTests, enter:
am instrument -w -e class com.android.demo.app.tests.UnitTests,com.android.demo.app.tests.FunctionTests#testCamera com.android.demo.app.tests/android.test.InstrumentationTestRunner

@tobykurien What should our <runner_class> be? I've tried all these without success:

am instrument -w com.SecUpwN.AIMSICD/Detection1
am instrument -w com.SecUpwN.AIMSICD.Detection1/android.test.InstrumentationTestRunner
am instrument -w com.SecUpwN.AIMSICD.test/Detection1
am instrument -w com.SecUpwN.AIMSICD/android.test.InstrumentationTestRunner
am instrument -w com.SecUpwN.AIMSICD/android.test.Detection1
am instrument -w -e class com.SecUpwN.AIMSICD.Detection1
am instrument -w -e class com.SecUpwN.AIMSICD.Detection1 com.SecUpwN.AIMSICD/android.test.InstrumentationTestRunner
am instrument -w -e class com.SecUpwN.AIMSICD.Detection1 com.SecUpwN.AIMSICD

References:
[1] "Testing from Other IDEs"
[2] "InstrumentationTestRunner"
[3] "Testing Fundamentals"
[4] "Activity Testing Tutorial"

[tobykurien]

The command that works is:

am instrument -w com.SecUpwN.AIMSICD.test/android.test.InstrumentationTestRunner

This will run all the tests. For just Detection1:

am instrument -e class com.SecUpwN.AIMSICD.test.Detection1 -w com.SecUpwN.AIMSICD.test/android.test.InstrumentationTestRunner

It's important to remember that the tests are run from a separate test application. It's in a separate APK, and hence has the .test added to its root package name. You need to install both the app and the test APK to be able to run tests. Gradle and Android Studio do this automatically for you.

[E3V3A]

So it doesn't install the test package automatically, when you install AIMSICD?
If not, how could we make it to do so, so that we can run these tests from the app itself?

I tried those commands, but they're not working. Probably because the test app hasn't been installed?
Where is it located and what is it called, after install?

[tobykurien]

It's a developer tool, not a user tool. Use Gradle or Android Studio to run the tests.

[E3V3A]

I know, but I'm using G and AS in a VM and the tests need to run on the app in metal I think. So if you know how to do this, please tell us, because we'd like to :

1. Run tests as part of the app, upon user request (when in a future debug mode, for example).
2. Run developer detection tests from command line using am.
3. Learn how to do this, since we're rather handicapped development wise, as you're already well aware of.

[SecUpwN]

@0x7678 and @larsgrefer, what are your thoughts on this? Results of #75 will be added here.