Sentry Android does not capture LogCat logs

[fzyzcjy]

Platform:

• Android -> If yes, which Device API (and compileSdkVersion/targetSdkVersion/Build tools) version?
• Java -> If yes, which Java (and sourceCompatibility/targetCompatibility) version?
• Kotlin -> If yes, which Kotlin (and jvmTarget) version?
• NDK -> If yes, which NDK/CMake version?
• React-Native -> If yes, which version?
• Timber -> If yes, which version?
• Log4j2 -> If yes, which version?
• Logback -> If yes, which version?
• Spring -> If yes, which version?

IDE:

• Android Studio -> If yes, which version?
• IntelliJ -> If yes, which version?
• Other -> If yes, which one?

Build system:

• Gradle -> If yes, which version?
• Buck -> If yes, which version?
• Bazel -> If yes, which version?
• Maven -> If yes, which version?
• Other -> If yes, which one?

Android Gradle Plugin:

• Yes -> If yes, which version?
• No

Sentry Android Gradle Plugin:

• Yes -> If yes, which version?
• No

Proguard/R8:

• Enabled
• Disabled

Platform installed with:

• JCenter
• Bintray
• Maven Central
• Manually

The version of the SDK:
3.x.x

---

I have the following issue:

The description goes here ...

Steps to reproduce:

• run sample

Actual result:
Sentry Android does not capture LogCat logs

Expected result:
when having things like Log.i(...), I hope sentry can automatically capture it.
thank you!

[bruno-garcia]

Thanks for raising @fzyzcjy. Last we looked at this it wasn't as straight forward as something like Timber. We'll do another investigation. Do you have suggestions on how to implement this and/or is willing to make a PR to add support?

[fzyzcjy]

@bruno-garcia I also do not have quite good solutions. If there is an idea I can make a PR. In addition, it is not possible to let every library use Timber. For example, one third party library may print out some Log.w or Log.e which I want to capture and recognize (because it can be the cause of some bugs), but it is not possible now :(

However, IMHO this is a very useful feature!

[bruno-garcia]

Now that we have attachment support it makes more sense to look into logcat again. It actually came up here: #1615

If you're willing to discuss it would be nice to discuss ideas. @marandaneto I'm sure will have opinions here btw.

Very high level it would be nice to have an options: attachLogcat and the SDK automagically sends logcat files cap'ed at some size on each event captured. We could build in parts but the final version of such features would propagate this flag to sentry-native which would also support it, and events from any layer would include the attached logcat file. We could eventually opt-in by default to this feature too.

[fzyzcjy]

@bruno-garcia Interested in the suggestion!

[marandaneto]

it'd be possible ootb only via bytecode manipulation or executing logcat params... and writing the output to a file or parsing the text, of course.
I'm not sure if it's a good idea though since it's rather a good practice to remove every logcat call from production apps via Proguard/R8 to reduce the app's size, there's some cost in it, but obviously, a nice feature to have.
I'm just questioning the usage of logcat in production Apps here to identify issues.

[fzyzcjy]

@marandaneto @bruno-garcia I realize this is REALLY VERY VERY useful, when I am debugging today. I am debugging a third party sdk, and finally one useful clue is found by examing system logcat. without using logcat to see what the 3rd party library logs out, I cannot find any useful clue.

[marandaneto]

We can execute Runtime.getRuntime().exec("logcat -v time"); and listen the buffer in a while loop.
https://github.com/pedrovgs/Lynx/blob/master/lynx/src/main/java/com/github/pedrovgs/lynx/model/Logcat.java

[fzyzcjy]

Sounds great! Will it require extra permission or slow down the app - inside release app builds?

[marandaneto]

Sounds great! Will it require extra permission or slow down the app - inside release app builds?

It does, but not for all versions apparently. https://github.com/pedrovgs/Lynx/blob/7b1f25172f2bd3f4563076dcd58ec532aebfb9f1/sample/src/main/AndroidManifest.xml#L20-L21
Well, ideally this is done on a background thread, but I don't have benchmarking data, it'd be an opt-in feature till we get proper feedback anyway.

[fzyzcjy]

Aha, that is for android 2.x if the comments are right. Then almost nobody is using 2.x :)

[kyze8439690]

Use code below to upload logcat including native crash happened

fun initSentry(context: Context, dsn: String) {
    SentryAndroid.init(context) {
        it.dsn = dsn
        it.tracesSampleRate = 0.03
        it.addEventProcessor(object: EventProcessor {
            override fun process(event: SentryEvent, hint: Any?): SentryEvent? {
                Runtime.getRuntime().exec("logcat -t 1000 *:I").inputStream.use { input ->
                    context.openFileOutput("logcat.txt", MODE_PRIVATE).use { out ->
                        input.copyTo(out)
                    }
                }
                return super.process(event, hint)
            }
        })
        it.isEnableScopeSync = true
    }
    Sentry.configureScope { scope ->
        scope.addAttachment(
                Attachment(
                        context.filesDir.path + "/logcat.txt",
                        "logcat.txt", "text/plain"
                )
        )
    }
}

[marandaneto]

https://developer.android.com/studio/preview/features#logcat

Not sure if the logcat output has changed or the IDE is parsing it, but worth checking it when working on this issue, worst case we have to parse 2 formats.

[marandaneto]

I suspect we can reuse this logcat parser from the Android framework itself
https://cs.android.com/android/platform/superproject/+/master:development/tools/bugreport/src/com/android/bugreport/logcat/LogcatParser.java

[buenaflor]

What approach would be most helpful - having breadcrumbs per log entry (only selected/filtered logs), having logs attached to crashes, or perhaps allowing both options?

[fzyzcjy]

More options looks more flexible IMHO. If possible, it would be the best to make this a library or a separate module, such that I can even gather those logcat logs and send to my own logging infra (e.g. when there is no crash but has warnings so I would like to have a look)

[markushi]

Having done our first feasibility checks (looking good! 🚀 ) we have a few options on how logs are collected and displayed on sentry.io
A) Collecting the logs and adding them as an attachment
This could be especially useful for crashes, to have a single file with the recent N log entries. Provides nice extra context, is only limited by attachment file size, but probably a bit hard to read and separated from the main sentry.io experience.

B) Collecting the logs as breadcrumbs
Probably easier to consume, but this could get very noisy and is limited (default breadcrumb limit: 100). We could filter the logs (e.g. only show breadcrumbs for Warnings and Errors).

I'm wondering if there are more options or even providing both could make sense.

@kyze8439690 @fzyzcjy it would be great to get your thoughts on this to help us shape this feature!

[fzyzcjy]

@markushi Congratulations on the checks! I personally prefer the former, since the latter is very limited in length.

In addition, I wonder whether it will be provided as a separate easy-to-use module? Then, for example, I can add a few lines (sentry_logcat_api_register_callback { log_message -> my_logging_service.add(log_message) }) besides the traditional Sentry initialization code, and then the logcat logs will be reported to my logging infra as well. IMHO, this will not cause heavy extra work, as long as the code has good modularization (which I think so)

[marandaneto]

I don't think we should raise error events because we'd not have stack traces anyway (making it less useful?).
I suggest adding all logs within the app as breadcrumbs, but having a sensitive min level, such as error, but configurable similar to how other logging integrations do: minBreadcrumbLevel = Error.
If it's still too noisy before can still filter out crumbs using the beforeBreadcrumb callback or configure a higher minBreadcrumbLevel level if any.

Using a file requires constant io, has to be done in a background thread, and is aware of concurrency issues.

The question is to try it out and check how verbose it is, compare the trade-offs and decide based on user experience.
Do we know any open-source app or library that is very verbose with logcat?

[kyze8439690]

We have transferred the logcat file to the sentry as an attachment. After describing the file type as 'text/plain', sentry can directly view the file content in text form on the web page without additional file download.

[kyze8439690]

I also tend to go for the first solution since we use breadcrumbs to log user behavior instead of app logs.

[marandaneto]

@kyze8439690 repos for context https://github.com/kyze8439690/logcatviewer/blob/master/src/main/java/com/github/logviewer/FloatingLogcatService.java

[buenaflor]

@kyze8439690 did you check on which Android versions your code running logcat during runtime works? After some checks this only works on API >= 23 and will reject if below that due to permissions.

[kyze8439690]

@kyze8439690 repos for context https://github.com/kyze8439690/logcatviewer/blob/master/src/main/java/com/github/logviewer/FloatingLogcatService.java

https://github.com/kyze8439690/logcatviewer/blob/master/src/main/java/com/github/logviewer/LogItem.java#L26

[kyze8439690]

@kyze8439690 did you check on which Android versions your code running logcat during runtime works? After some checks this only works on API >= 23 and will reject if below that due to permissions.

Sorry, we didn't do a complete compatibility test because the logcat file upload is not for end users. But I think it is acceptable to support only API level 23 and above.

[romtsn]

We've shipped this. See https://docs.sentry.io/platforms/android/configuration/integrations/logcat/

[kyze8439690]

How many lines of logcat can be recorded right now?

[buenaflor]

Our logcat integration only includes creating breadcrumbs out of logs with a specified minLevel which you can configure in your build.gradle (which also includes logs by third-party libraries since our implementation is based on bytecode manipulation).

If you still want to log the last n number of logs as an attachment you can still do that as you currently do by adding an EventProcessor and injecting a logcat shell command.

[kasogg]

Is there a way to manual install Logcat Integration? I don't want to install with the Sentry Android Gradle Plugin. @buenaflor

[romtsn]

@kasogg You would have to manually replace all of your Log.e/d/w... calls with SentryLogcatAdapter.e/d/w..., though the class is marked as Internal, so we do not guarantee api stability.

We can reconsider though, if there's a good usecase. Are you fine with manually replacing the log calls? We could make it officially public.

[kasogg]

@kasogg You would have to manually replace all of your Log.e/d/w... calls with SentryLogcatAdapter.e/d/w..., though the class is marked as Internal, so we do not guarantee api stability.

We can reconsider though, if there's a good usecase. Are you fine with manually replacing the log calls? We could make it officially public.

My requirement is:

1. Only when the user agrees privacy policy can app upload the logcat message.
2. I want to upload not only app logs but also system logs(Not necessarily)
3. logs upload only when Crash/ANR

I find a workaround:

options.addEventProcessor(object : EventProcessor {
     override fun process(event: SentryEvent, hint: Hint): SentryEvent? {
          if (event.isCrashed || event.exceptions?.any { it.mechanism?.type == "ANR" } == true) {
                                    Runtime.getRuntime().exec("logcat -t 1000 *:I").inputStream.use { input ->
                                        application.openFileOutput("sentry_logcat.txt", MODE_PRIVATE).use { out ->
                                            input.copyTo(out)
                                        }
                                    }
                                    hint.addAttachment(Attachment(application.filesDir.path + "/sentry_logcat.txt", "sentry_logcat.txt", "text/plain"))
                            }
                            return super.process(event, hint)
                        }
                    })
                        

[romtsn]

@kasogg ok, it's a bit different usecase from what we have implemented, you can open up a new issue if you will and we'll see if it gets more demand we might eventually implement it, but for now I'd suggest you stick to your own solution. thanks

[buenaflor]

@romtsn This was the initial proposed solution but it only works for API level >= 23, that's why we opted for the bytecode manipulation route