Secure Setting Logger

Share and discuss your flows and ideas with other users.

Moderator: Martin

Post Reply
User avatar
Posts: 2478
Joined: 21 Jul 2017 17:50

Secure Setting Logger

Post by Desmanto » 05 Mar 2019 15:18

How many times you found some setting you want to Automate using Automagic and can't find the corresponding action to change it? Or maybe you are sick with Google that keep limiting android generic function and making things hard, such as changing network mode/location now require root in newer android version. Or you are using custom ROM that has some special setting that doesn't have the usual toggling method. Even if you have root, you probably can't change the value automatically. Those above can be done using Control UI, which I have explained here : viewtopic.php?f=6&t=7320 Unfortunately, Control UI has UI distraction and doesn't work when the screen is off/locked. So in here, we are going to play with one of the powerful, yet dangerous part of the Android System, Write System setting. (also called write secure setting)

System setting database contains most of the setting from generic to secure ones. Network mode, display brightness, volume and many various settings are stored in here. There are 3 categories until Android Pie 9.0, which are (from android documentation)
System : System settings, containing miscellaneous system preferences. This table holds simple name/value pairs. There are convenience functions for accessing individual settings entries.
Secure : Secure system settings, containing system preferences that applications can read but are not allowed to write. These are for preferences that the user must explicitly modify through the system UI or specialized APIs for those values, not modified directly by applications.
Global : Global system settings, containing preferences that always apply identically to all defined users. Applications can read these but are not allowed to write; like the "Secure" settings, these are for preferences that the user must explicitly modify through the system UI or specialized APIs for those values

Prior to Marshmallow 6.0, these system setting is stored in SQLite db, /data/data/
After Marshmallow 6.0 above, the format has changed to xml and store in /data/system/users/0/settings_global.xml, /data/system/users/0/settings_secure.xml and /data/system/users/0/settings_system.xml
(My flow here will use the Marshmallow 6.0 above format, since I have Pie 9.0 now.)

Purpose of the flow
Since these setting database contains one of the setting that you can change, it is curious to us that which setting name (key) correspond to the setting we want to change. Example, location feature is usually stored at Secure Category with key : location_providers_allowed and value : network,gps. How we supposed to know that? Majority of the key value will be similar across multiple android devices. But with so many various OEM/brand implementation, some of these value can have different names or located in different database. Not to mention any special feature for certain vendors, they can have some unique naming.

Forget about the detail of the key value, just remember any setting you want to find/use, will most likely fall into one of these 3 categories. So the purpose of this flow, is to help you find the corresponding key value that you can use to change the setting later. Example, to find that location_providers_allowed is the one responding to set the location feature.

Warning : Enabling adb, and using Action Set System Setting improperly can brick your phone. You may lose your data and may need reset your phone. There are cases of bootloop caused by improper modification to the system setting. Make sure you understand the risk before continuing. When in doubt, just stop. Do With Your Own Risk.

In order to use this flow, you must have granted Automagic the write secure setting permission. If you have grant it (or you have root), you can skip to Flow Import directly.

PC setup
You need adb for this, of course means a PC. Can be windows (preferred), linux or mac. No root is required, that's why we need the help of adb. Xda has a quite neat tutorial for this : How to Install ADB on Windows, macOS, and Linux. You should check it out.
For windows, the fastest way to install the adb is to use adb installer. Just need to run it as admin, yes yes yes, and it is done.

You might to need to install your phone specific driver to make sure the adb can recognize your phone. My previous phones are in this category, need to install additional phone driver provided in the internal emulated CD built-in. Do whatever necessary until adb can detect your devices. You might need to disable firewall/antivirus for a while during the adb command, if the firewall/antivirus policy is too tight.

Phone setup
At your phone, make sure the USB debugging is on, which is hidden in developer options. To enable developer options (if you haven't), tap 7 times at the build number/kernel version at about phone. After that, go to settings > Developer option > enable the USB Debugging, tap OK for the prompt. Developer option can be located on different place, depends on the phone/ROM.

After adb and drivers has been installed properly, and USB debugging has been turned on, connect your phone with the PC using the original cable (or any cable that support data transfer). For some phones, you might need to switch to MTP to enable the USB Debugging detection. You might see some drivers installing if it is the first time you connect the adb.

Command Prompt
At PC, open command prompt/terminal at the adb folder (or can be anywhere if you install system-wide). Type this command to check if your device has been recognized

Code: Select all

adb devices
You should see the some 8 hexadecimal that is your device adb serial number. There will be a prompt to allow the USB debugging connection for this PC, tap OK at the phone.

Then type this (or just copy from here, right click at command prompt > paste) to grant automagic write secure setting permission

Code: Select all

adb shell pm grant ch.gridvision.ppam.androidautomagic android.permission.WRITE_SECURE_SETTINGS
After you enter it, there will be nothing display to indicate it is success. You can only know if it errors, there will be error code displayed. So if nothing appear, you are good to go. But if you just want to make sure, you can check it using dumpsys, make sure granted=true.

Code: Select all

adb shell dumpsys package ch.gridvision.ppam.androidautomagic | grep WRITE_SECURE
You should see something like this.
lineageos.permission.WRITE_SECURE_SETTINGS: granted=true
android.permission.WRITE_SECURE_SETTINGS: granted=true

The permission has been granted, now we can continue on the phone. Eject/unmount any mounted drive from you phone and unplug the cable. PC is not needed anymore.
You only need grant the permission once. The permission sustained until you uninstall automagic, factory reset your or change ROM.

Grant Write PC without PC
For Root users, you have the privellege. You can simply run the command above directly from terminal emulator with root privellege. (without adb shell)

Code: Select all

pm grant ch.gridvision.ppam.androidautomagic android.permission.WRITE_SECURE_SETTINGS
Or you can also use Automagic action : Execute Root Command, copy-paste the command above, execute it manually once and done.

Flow import
Download this flow and import it : Secure Setting Logger
You will have 2 flows, Secure Setting and Secure Setting Logger. Just enable Secure Setting flow only, add the shortcut to the home screen for easy access. Secure Setting Logger flow will be enabled by using Secure Setting flow choices later.

In order to start logging, you have to create the glovar first. I use the glovar name global_secure_setting, create it manually with type list, leave it blank, OK. Or you can create it using script, execute below script manually.

Code: Select all

global_secure_setting = newList();
This is to store the logged glovar later. For root user, you should create another folder under the /storage/emulated/0/Automagic/Resources/, because this is where I store the sec.txt, the result of the query. You can of course change the save location, change the script also.

Flow branches
Secure Setting Logger flow is a single branch, only for logging purpose. So it is easy to understand. But the main flow Secure Setting, has 5 main branches (and 2 sub branches). For non root user, you can only use the 3 branches on the right, which is Start Logging, Stop Logging, and Show Last Log. Root user can use all of them.
1. Secure Setting Flow.png
1. Secure Setting Flow.png (355.95 KiB) Viewed 4054 times
Unfortunately, until Automagic action Init Variable System Setting can support global pattern or all settings, there is only root method to query all available settings. I use the cat /data/system/users/0/settings_global.xml (secure and system) method, instead of the built-in settings list global. Because xml can provide more detail data, there is default value and the last app which modify the value. But currently I only use the key and value, so default value and last app are useless for now. Despite that, I still keep this method in case I am going to expand the flow later.

How to use
For non root user, simply tap the shortcut you have created. Choose Start Logging for the first time. (make sure you have create the glovar before). This will enable Secure Setting Logger flow, which will start logging all system setting changes. Go to the setting of your phone and change something, example change the network mode for SIM 1. You should see a toast message containing the [category, key, old value and the new value]. Toast message only indicate the changes, you can check the changes later. Change as many thing as you want/need and see the toast message.
2. Change Network Mode.png
2. Change Network Mode.png (206.96 KiB) Viewed 4053 times
Once you have finished, tap the shortcut again and choose Stop Logging. This will disable the Secure Setting Logger flow and show you all changes that happens since you started to enable the flow. You can choose any of these and you will be presented option to copy category, key, value from the choosen. Tap any of these to copy it to clipboard. Usually you will need the key only, and sometimes value. You can remember the category easily. You can use this value later in action Set System Setting. In case you mis-closes the input dialog, you can still reopen last log by running the flow again and choose Show Last Log.

The logger flow has been set to stop after 20 executions. This is to prevent the logger flow running forever. You can increase this by changing the execution count. You can also change the maximum last log stored (default 20).

Root branch
For root user, besides above feature, you can do more. You can choose Save All Settings, this will query all available setting using root command and save it to /storage/emulated/0/Automagic/Resources/sec.txt. The sec.txt is json, in form of nested map-list, with the key is the setting name and the list are [id, value, packagename, default value, and category].

Once you save the setting, you can use it for future comparison. Use your phone as usual, change some setting and run the flow again. Now choose What Changed? This will query current system setting and compare it to the sec.txt we have saved before, find out the changes and show it. You can choose any of those value to copy the category, key, and value. This is very useful for some setting that you don't know when it will change and you don't want to run the logger flow all the time.

Another cool feature you can use is to Show All. This will show all key value including the category it belongs to. You can also choose to Show Global, to show only global category; the same for secure and system. Last, you can even use Search keyword feature. Put in the keyword you want to find, and it will query upon all key or value and show you the matched ones. This is extremely useful if you already know part of the keyword. Searching in each one of the category require 3 searches, but using this only requires single search. However, please remember that all option from Show All, Search keyword, show global/secure/system; use the sec.txt you have save before. So you have to run the Save All Settings at least once. And remember that these choices doesn't show current setting value. To show current one, you have to choose Save All Settings once again before using any of these 5 choices. I purposely make it this way, so you don't query root command too much to the system. Because usually we only need to save once, but can search for several times.

Some catch for Oreo and Pie
For Android Oreo and Pie users, before you start logging, maybe you should turn off Auto brightness (Oreo) or Adaptive Brightness (Pie). Both android version will change the value too frequently if you are in the dynamic lighting condition or you cover your light sensor when using your phone. You will understand it if you don't turn it off and see so many toast message indicating the changes. If you look at my log, you can see half of the log is "contaminated" by
[System] screen_brightness changes.

Using value found
After you got the value, you can copy from the logging result, you can then add the Action Set System Setting, choose the category it has shown to you and paste/search the key/value you've got before. Since it shows you the possibility of the value too, you can try to mimic the same value. Example, [secure] location_providers_allowed, can have value : network when only network location provider battery saving mode enabled. When the GPS is on, the value changed to network,gps You can then use +gps in here to add the gps value to this key. Or -gps to remove the gps. This is the same as turning on/off the GPS state. Remember to do it this way, as you can only change one value at one time. Some phone/ROM allow the changes more than one at once, but if your phone/ROM doesn't support it, change only one value at one time. Below is example from the network that we have got from previous test.
3. Set System Setting.png
3. Set System Setting.png (455.92 KiB) Viewed 4053 times
For other value, sometimes it is single digit number. Some are toggling value, such as 0 and 1, maybe 0 and 2. Some can be choices of number, can be 0, 1, 2, or 3. Some have packagename inside. Just mimic whatever state you got from the changes. You can also use this in Trigger System Setting Changed, to monitor the value of this setting. Example, you can monitor if gaming mode has been enabled, then you start the timer to count how long you enable the gaming mode.

However, remember that, not all device/ROM honor/reflect the changes you made directly to the system setting database. Sometimes you still have to open the setting page which contain the setting you are trying to change (for network mode, have to use launch app to go directly to that setting page). As explained above, some setting are protected by special API or require some UI interaction to get the changes. For those kind of setting, we are out of luck. We can't change the value directly, only can act on the changes of the value (trigger System Setting Changed still works). Control UI still works though, but yeah, hindered by UI distraction. Example, global - sysui_tuner_demo_on, which correspond to the system UI demo mode. When we enable Demo mode, this will be changed to 1. However, changing this value from 0 to 1, won't enable Demo mode. Because demo mode must be enabled by using special broadcast, with android.permission.DUMP or root privellege. sysui_tuner_demo_on only store the state of the demo mode, but doesn't reflect the changes upon it.

Adb shell List settings
For root user, some of you might be asking why I don't use built-in settings root command to list out all possible system settings

Code: Select all

settings list global
settings list secure
settings list system
It is because I don't know this when I create the flow! After I learnt this command, found out it only provide key=value only. But parsing from xml, can give the last packagename modifier and default value. So I just use my own previous finished method, rather than recreate the script again to parse the result of the command.

LineageOS System Setting
Beside android built-in System Setting, custom ROM LineageOS (LOS) also has its own separated System setting. As per LOS 16, the setting is stored as SQLite db at /data/user_de/0/org.lineageos.lineagesettings/databases/lineagesettings.db. Any LOS based ROM also will have this. Example Resurrection Remix (RR) which is based on LOS, also has this database. You can check that file, if it exists, then your ROM support it.

Automagic support to Init and Set the LOS system setting too. You need to grant additional permission from adb or terminal root.

Code: Select all

adb shell pm grant ch.gridvision.ppam.androidautomagic lineageos.permission.WRITE_SECURE_SETTINGS
As per LOS 15.1, the actions are still working. However since LOS 16, and until Automagic 1.37.0, Automagic can't recognize it anymore. I am using RR 7.0.0 Pie 9.0 based on LOS 16, Set LineageOS System Setting now give me error : "Changing LineageOS system settings is not supported on this ROM". So we should wait until maybe LOS fix it or maybe Martin add support for LOS 16.

We hope to get the similar action set as in default android System Setting too. I can't built the flow for it, since there is no trigger to detect the changes. But since LOS based user are minority, even less than the percentage of root users, so this maybe not such a pressing need to support it. I just document this here for completeness.

Hopefully, this flow will help you to find the correct system setting you can modify to help in creating another flow. However, remember to be careful when changing these value. Make sure you have made proper backup first, either your other data, the setting or Automagic's flow backup (you can backup your flow using my Automagic Backup Flow : viewtopic.php?f=3&t=7858). Changing/deleting wrong value can result in catastrophe, as I have experienced here : viewtopic.php?f=4&t=7272

At last, Happy Logging! :D
Index of Automagic useful thread List of my other useful posts (and others')
Xiaomi Redmi Note 5 (whyred), AOSP Extended v6.7 build 20200310 Official, Android Pie 9.0, Rooted.

Posts: 57
Joined: 17 Apr 2017 05:31

Re: Secure Setting Logger

Post by tphg » 06 Mar 2019 00:58

Always see academic post from you. Thanks @desmanto for the useful tricks.

Posts: 25
Joined: 15 Jul 2015 19:42

Re: Secure Setting Logger

Post by moonspeak » 28 Mar 2019 20:52

This is fantastic! I have wanted this exact functionality for a long time. Needed it so many times. But I somehow never thought about copying the settings files to compare changes (guess I didn't realize they simply were files like that).

Thank you so much for sharing your work. I can't wait to use this! I agree with tphg; you're always posting very in-depth and informative stuff. And it is appreciated! :)

Posts: 364
Joined: 24 Jan 2015 02:06

Re: Secure Setting Logger

Post by anuraag » 08 Nov 2019 23:38

I request you to create similar for logcat when 1.38 out of EAP. You might have seen tasker's latest beta feature.

User avatar
Posts: 2478
Joined: 21 Jul 2017 17:50

Re: Secure Setting Logger

Post by Desmanto » 10 Nov 2019 18:25

anuraag wrote:
08 Nov 2019 23:38
I request you to create similar for logcat when 1.38 out of EAP. You might have seen tasker's latest beta feature.
I am actually about to request the feature. Currently I use a simple root command logcat with 3 seconds before to log certain action. Then parse it to get the event and filter it catch that. So far, I only use it for my fingerprint sensor (trigger fingerprint gesture doesn't work at my RN5).

I can extend the result to get a similar UI with what tasker has done there (including finding keyword). The only problem is I am afraid that if the Command Output trigger is enable all the time, it will consume much more battery, as it keep looking at the logacat to filter out some words. I don't know how tasker deal with it under the hood. Maybe Martin can give us some clue. When I have some time, I will test the command result and check if it drains battery significantly.

And even if 1.38 is out, I don't know if the execute command alone (without root) has enough permission to get the read log permission (in order to logcat). Maybe we need a new trigger specifically for this, similar to tasker. And in this case, I don't need to create the flow anymore :)
Index of Automagic useful thread List of my other useful posts (and others')
Xiaomi Redmi Note 5 (whyred), AOSP Extended v6.7 build 20200310 Official, Android Pie 9.0, Rooted.

Post Reply