Debug: Need on screen logcat entries

[E3V3A]

In order to better debug the detection parameters, we'd like to have some of the log.i and log.e entries displayed on screen. We're particularly interested in ones coming from these files:

AIMSICDDbAdapter.java
Cell.java
CellTracker.java 
Device.java
DeviceFragment.java
DeviceApi17.java 
RilExecutor.java
SamsungMulticlientRilExecutor.java
SmsReceiver.java

Perhaps it is possible to add it in such a way that it is only shown when built with Gradle's "assembleDebug"? Also, when they're shown, it would be extra nice if it was shown just like the on-screen-dispaly (OSD) in the Android built-in CPU and process info...

@tobykurien What do you think? Is this possible?

Q: Why all this?
A: We need a way to debug individual items in the detection matrix.

[SecUpwN]

@tobykurien, since you are our main coder, would you please have a look one of my favorite Apps, the VLC Media Player for Android by @videolan? I just examined their error logging yesterday (navigate through Settings into Error Protocols). I like that it continuously logs all errors in the background whilst running the App and features the ability to come back to that screen to copy the whole logcat through the touch of a button. Is this (or something similar) implementable into our own App? ;-)

[He3556]

A simple first step can be a function to write all cell-data into a file.txt
if (detection=true)

[SecUpwN]

@He3556, sounds good. But have you seen how VLC for Android does the whole logging thing? I would like our App to feature this the same way. After that, we can implement the text file logging..

[E3V3A]

Just to bump this. Have you guys seen how its done on the built-in (on Samsung at least) OSD process/CPU usage list? If it's there, surely there are already sources for this out there somewhere. Can we find some example code for this + screen-shot, to post here, to move this ahead?

[SecUpwN]

Pushing this further: Have you tested how VLC for Android does on screen logcats, @E3V3A? I've made some screenshots below in case you where "too lazy" to look into it. How do you like that navigation?

From the above screenshots you can see how a user has to move through the menu entries to reach the on screen logging. Once enabled, it always runs in the background and logs whatever might cause an error. I've already written an E-Mail to the folks at @videolan to provide me with sample code on how to best include this in our App, but so far got no response. Source code is available via tarball and Git.

[E3V3A]

Yes, that looks good. Originally I was thinking to use the built-in on-screen-display (OSD) as found on here: Settings >> Developer options : Monitoring

Show CPU usage [x]
Screen overlay showing current CPU usage

There is an app called CPU Usage Overlay that uses this feature. But I'm not sure how compatible this is on other devices.

So we can use it in such a way to produce a filtered logcat directly on top of screen. This is great for live debugging, but makes it hard to see and possibly use.

Another variation of this is the App called Log HUD - Live Logcat Overlay:

[SecUpwN]

@E3V3A, I am voting against such overlay logacts. People cannot copy and paste them conveniently and also might submit false logcats when re-typing by hand. Personally speaking, I would like this whole thing to be exactly like VLC implemented it. That way, we could enable and disable ongoing log collection and would also have a button for copying the whole thing. Adding another button to send it out via E-Mail or directly open an Issue + Logcat on our GitHub would be the cherry on top of the pie. 🍒

[E3V3A]

@SecUpwN It's not for other people, it's for development only. Problem is that beacuse of what we're doing, we have very chatty logcats, missing any live sense of when stuff happens and why. This would help see what happens at the time it happens, then it's easier to go back to logcat file (as you descrive in VLC method) and see where it happens. For example when testing type-0 pings.

Some of the code for that AOS function is here: DevelopmentSettings.java.

[SecUpwN]

@SecUpwN It's not for other people, it's for development only.

I perfectly know. But since we will be developing quite a long time (and detailed logcats are useful), I consider it better to fully implement this. People will be able to contribute better bug reports instead of the crappy ones that we currently have piling up. In my eyes, this feature is very important, not just for testing, but also in the long run to support and improve our App on as many devices as possible.

[E3V3A]

@SecUpwN But you've already opened issue #138 for that!?

[SecUpwN]

@SecUpwN But you've already opened issue #138 for that!?

Partially right. While you would like on screen logcats and I also see the need for better logcats, implementation of the above proposal like VLC should essentially solve both of these Issues.

[He3556]

Yes SecUpwN, i am totally with you. Is this No1 priority right now? I hope so - because it would also help to check the detection mechanisms. Is somebody working on it?

[SecUpwN]

@He3556, I have been crawling the interwebz for more valuable developers like @mrmaffen to support @tobykurien, because I do not want him to leave our project for being overloaded with work. But surpise, surprise, they are either busy, think that they lack the knowledge, or simply have "not time for other projects". Is there some sort of organization like @guardianproject which develops Android Apps for Privacy which I should be explaining our struggles? It really is a shame that we have so many followers and forks, but close to zero pull requests. Tell me what to do or where to continue my hunt.

[SecUpwN]

@E3V3A, have you been testing the new Debugging menu entry in the current version yet? I really like that it also can be sent via E-Mail! Does this solve your Issue here? Some questions to @tobykurien:

1. Can you please fix that the debugging logs do not move up and down and so forth?
2. Are all logs of these files being tracked as well? Can we debug individual items somehow?

[tobykurien]

@SecUpwN press the Stop button to stop the scrolling, and Start to restart it. Any logs spit out by the application will show up there. If you want to add filtering, add that as a separate issue.

[SecUpwN]

@SecUpwN press the Stop button to stop the scrolling, and Start to restart it.

Thanks for the suggestion, @tobykurien. It actually doesn't help, the scrolling keeps moving up and down after that as well. About the filtering: @E3V3A wanted the logger to display log.e and log.i:

AIMSICDDbAdapter.java
Cell.java
CellTracker.java 
Device.java
DeviceFragment.java
DeviceApi17.java 
RilExecutor.java
SamsungMulticlientRilExecutor.java
SmsReceiver.java

Shall I add a new separate Issue for the above things, or is it good enough if I mention it here?

[E3V3A]

I've now tested the Logging and it doesn't quite seem to work on my device. The log is there, but I cannot scroll and and all the buttons shown, doesn't seem to do anything. The app basically stop's responding while this is turned on. I'd like to know how this was implemented and what it's doing. Please don't point me to any code, just tell me in words with a couple sentences, what it does. (Where are the logs coming from? etc)

[tobykurien]

It runs logcat in a shell process. If there are tons of logs, it can hang the app. To be honest, I'm not willing to spend time basically replicating logcat functionality - I feel it's a total waste of my time, so things like filtering, etc. are not things I would work on - the dev tools already provide all that. I knew that adding this basic screen would end up with 10 other issues related to it. Feel free to add them, hopefully some other developer will tackle them.

[E3V3A]

@tobykurien At minimum, thanks a huge bundle for putting this in there. I hear you and agree completely, I also knew there would be more issues popping up with this. We'll look into it. Don't spend any more time on this! There are plenty of more important stuff to work on, as apparent from #206 and related ones. The log stuff can be done perfectly well with my command line logcat dump function. My logcat is very chatty even on MID settings, so what you mentioned elsewhere, that large and fast logcats could be problematic, applies to me directly.

@SecUpwN @He3556 @andr3jx Let's try to avoid pushing dev's to implement new stuff, until old bugs are fixed. (It obviously applies for me too.)

[SecUpwN]

@E3V3A, so what are we actually missing until this Issue is finally solved?

[E3V3A]

The issue is that we're trying to see a limited logcat in our app, that can already be seen MUCH better in several other logcat related apps, and uploaded/saved etc. and the Android command line logcat work just excellent. So we're wasting time implementing a useless and buggy feature, which is not even what I asked for, when originally opening this issue. The idea, which was to to only show logs of a few specifically labelled tags, OSD (or similar) style. As an example, see RF Signal Tracker and enable debug messages. This was supposed to be a developer feature.

[SecUpwN]

@E3V3A, true point. But we now have that feature in our App and even if we don't need it now, we might later. What do you suggest? Full remove it after all the hard work? I feel that we've been putting enough work into it that it deserves to stay for good. So to sum up why this Issue here is still open, how about a final pull request to only show logs of a few specifically labelled tags as in RF Signal Tracker?

[He3556]

i really like the debug log and i am using it all the time. Please don't remove it.

I vote for a output like this in the main view where we can see what the app is doing. At least the things that have to do with detection. I miss this in all the other apps we know. A widget would be perfect - but as long as we don't have that i want to keep the debug log.

[Ueland]

@E3V3A, how do i activate debugging in RF Signal Tracker and where can i see what it logs? (Like you want it)

[E3V3A]

Settings >> Error messages

[E3V3A]

If there's an error, it pops up.

[SecUpwN]

If there's an error, it pops up.

Any way to produce one just so that @Ueland can see what it should look like?

[Ueland]

Ah, got it. It should not be a problem to implement the same feature in AIMSICD with the current logging system. (Log everything as before, if level =>error, show pop up).

[He3556]

Is it ok for you guys when i add my log output to the AIMSICDbAdapter in the repro?
like:

   Log.v(TAG, "Cell updated in local db: " + cellID);
   Log.v(TAG, "New Cell insert in local db: " + cellID);
   Log.v(TAG, "Cell already in OCID DB (update): " + cellID);
   Log.v(TAG, "Cell deleted: " + cellId);
   Log.v(TAG, "LAC checked - no changes: " + cell.getCID() + "-"+ cell.getLAC());
   Log.i(TAG, "CHANGING LAC!!! - CID: " + cell.getCID() + " Current LAC: " + cell.getLAC() + " Database LAC: " + cursor.getInt(0));

I think this has some importance for us all - especially when a false changing LAC is happening again.

[He3556]

Are you (@eva) talking about improving the debug log in our App? Just a few hours ago you wanted to delete it - now you want to invest even more time on it? I am confused - we need to get the signal strength detection finished and work on the Neighboring Cell detection.

[Ueland]

@He3556 Yes, i would like for those log entries to get into the app, as i have problems running both your version and the one i keep building myself :) (and i keep getting LAC warning daily now)

[SecUpwN]

@E3V3A, @He3556 and @Ueland, is this solved now after 74704c6 has been added?

[Ueland]

I would say so, yes.

[SecUpwN]

I would say so, yes.

Thanks, @Ueland, I agree with you. @E3V3A, since you've created this, please close when done.