Variable Logger

Share and discuss your flows and ideas with other users.

Moderator: Martin

Post Reply
User avatar
Desmanto
Posts: 2203
Joined: 21 Jul 2017 17:50

Variable Logger

Post by Desmanto » 12 Mar 2018 18:12

Prologue
How many times we stumble upon a trigger or condition or action; and we want to know what are the variables that element provides? Using debug dialog is not enough, especially when dealing with trigger. Because the flow only get executed when the event triggered, and we don't have the time to look upon the flow (debug dialog) everytime it happen. Wouldn't it be easier if we can log those variables and look upon them later?

Introducing : Variable Logger
This flow will log all of available variable, excluding the glovar (since we can check it already), flow_name (because we already know it), and trigger (usually we have only one trigger to log).

Before we start, we should create a glovar with the type list. We are going to add the log to this list, all variables will be logged to this glovar. Since it is a one-time job, I usually create it manually. Go to Hamburger menu (swipe from left) > Global Variables > + sign (Add symbol) > type in the glovar name we want, in this case, global_whatsapp > OK > Change the type from String (default) to List > OK. You should see a new glovar : global_whatsapp with empty value.
01. Creating Glovar.png
01. Creating Glovar.png (344.39 KiB) Viewed 2660 times
There is actually another way to create it, and require a single line of script only. Just open the flow and go the element at the top right corner, open the script and manually execute from there. The script

Code: Select all

global_whatsapp = newList();
02. newList.png
02. newList.png (186.33 KiB) Viewed 2660 times
will create a new list with the name global_whatsapp. I usually hate this kind of one time use script, but this same script can be used to empty the glovar in case you need it. So rather than clicking so many times, you simply open this script and execute it; it will empty the glovar.


Main Element
The main element of the flow is just simply a single script after the trigger :

Code: Select all

//get all variable names
vartemp = getVariableNames();

//remove global variable, {flow_name} and {trigger}
var = newList();
for(i in vartemp)
{
  //remove |trigger if you use multiple triggers
  if(NOT matches(i, "global_.*|flow_name|trigger"))
    addElement(var, i);
}

//create new map to store the list of variable
map = newMap();
for(i in var)
  addMapEntry(map, i, eval(i));

//add the map to the selected logging global variable
addElement(global_whatsapp, map);
I will explain the script, in case you need to modify it to suit your needs.
1. First part of the script is gather all available variables into vartemp. Glovar is included.
2. We don't need the glovar, as we can check it by ourself. Thus we need to remove it. flow_name also not needed. trigger usually not needed. But in case you are logging a flow with multiple trigger, you should remove |trigger from the script, so you can log it too. Example when you are logging 6 trigger of device orientation in single flow. So from vartemp, we move only the variables we need to var.
3. Additional info on the matching, you can modify this part to suit your need. NOT matches() can be changed to matches(), so you only log certain variables, instead of all variables. Example, if you want to log only the battery level and the time, you can change the NOT matches() line to

Code: Select all

if(matches(i, "triggertime|battery_level"))
4. Third part is the mapping part. The var now only consist the name of the variables, but not the value. To get the value, we need to use getValue() or eval(). I use eval(), since it is shorter for this case. So from each variable names in var, we create a key using the name and value assigned from the real value of each variable. The result is the map contains all variables mapped to their values.
5. We add this map to the glovar list we have created before. In this case I am logging whatsapp notification, so I use global_whatsapp. You can change the name if you want, just make sure you have create the glovar first as shown above.

To sum up, the only thing you need to change is only the last line of the script. Change the global_whatsapp to other name you have created to log for certain flow.
You can copy this element to any where of your other flow and log the variables from the beginning, middle, or the end of the flow; depends on where you attach this element. Just make sure you have created the corresponding glovar before.

Execution Count protection
After the script, you can see a condition Execution Count. This is to limit the number of trigger can happen before disabling this flow. The reason we need this is because sometimes a flow can be triggered so many times and we forgot we have enabled this flow. I have ever trying to log cell id, so I use the core script only (no execution count yet). I enable the logging and forget it afterward (distracted by other things). Then a day later, I found out Automagic somewhat kinda slow down or laggy in certain part. When scrolling and checking my flowception, I just realized I have enabled the cell id flow logging. When I check the glovar, I have thousands of the list !!! No wonder everything slow down and seems to drain more battery.

By using execution count, you can limit to certain execution before stop logging. You can set to any value you need. I just make it 50; 10 is actually enough for most cases. But sometimes you need more variation in your logged data. Depends on your need, you maybe can disconnect this element; if you need unlimited data. Just make sure you remember when to disable the logging flow, as it will create thousands of elements in the glovar list when you neglect it.

One thing to note for the Execution Count. This element has its own internal counter, which can't be seen (I can't find any way to check it). There is also no direct method to reset the counter. (maybe should request these features). So after this got executed for 10 times, it will stay as 10. In case you want to reset it for the next logging cycle, you should edit this, change it to "Timed", duration 1s, save. Open it again, and uncheck the "Timed" and save again. This will reset the counter to 0 again. So you next logging cycle won't be stopped after 40 execution only (if previous counter is 10).

Trimming Logged Glovar
I have explained the first separated script to create the glovar, 4 core elements of the flow. The last is the separated script at the bottom. This is to trim the list. After logging for a while, we have gathered too many data. we want to clean up the glovar, but we still want to retain the last data created. You can go to the glovar and delete the elements you don't need anymore. But imagine doing this for hundreds of elements!!! So the easiest is to use script and remove the all elements, except the last 5. (we want to delete all of them, but we still need some samples). The script is just :

Code: Select all

removeElements(global_whatsapp, 0, length(global_whatsapp) -5)
Replace the -5 to the number of elements from the last, those you want to keep. Simply execute manually this script (just like the new glovar) and you have deleted those hundreds of unneeded elements, except the last 5.

Flow FEP and AES
For FEP and AES, I just leave the FEP default to parallel, because most of the time you are logging for a very quick event or a very long intermittent event. The script should execute very fast that even parallel execution of multiple triggers shouldn't cause any problem. But in case you want to log it sequentially or skip certain event, you should change the FEP accordingly (to wait or skip).

AES should be no problem as well, default 60 should be enough. This flow only cost 2 elements perexecution (script and execution count), so you can have 30 triggered event per minute. In case you need more, especially when logging multiple quick trigger, you can always increase it by yourself.

Checking the Glovar
After you enable the flow for a while, and you know you have triggered the flow several times (several people sent you whatsapp); you can go to the Glovar to check the value again. This is the example of the logging.
03. Glovar.png
03. Glovar.png (159.38 KiB) Viewed 2660 times
The glovar will contains the list of elements of the triggered event. You can tap on each element > Change value. You can see all available variables from that trigger event, along with the value they hold. You can tap on each value to see more detail. Or if the variable is a list, you can dig down even more, and so on.

Further Checking
You can even use the logged variables to execute another flow. But I haven't finish yet. You can combine them to csv, exported it and open it in spreadsheet for better view. For meanwhile, you can use this script to combine it (not included in the flow).

Code: Select all

list = global_whatsapp;

csv = newList();
addElement(csv, "{getMapKeys(list[0]),listformat,comma}");

for(i in list)
  addElement(csv, "{getMapValues(i),listformat,comma}");
  
csv = "{csv,listformat}";
Replace global_whatsapp with the glovar you use. csv will contain the logged variables in csv format. You can put this {csv} in the Text Field of action Write to File, to export it as csv. You can even put extra script to set the glovar to empty (after exported), so it will start collecting from beginning again; not burdening the whole glovar.

There are some catch though, especially when your variables contains special character, comma and the kind. I haven't found the proper method to deal with it, thus haven't put this in the flow. I planned to combine the function later in the next complementary flow (probably need a long time).


Main Usage
The main usage of this flow is to collect data, log data and find out what variables available from a flow/trigger and what are the values of the variables. This logging mainly to help you in designing the flow, to give you idea what are the data available to be processed. It can be used for troubleshooting too, but probably debug dialog is more useful for that usage.

You can log something that happens too quick to be checked using debug dialog. Example : Device orientation triggers. Or you can log something that happen in a very long time span, that you mostly can't check it using debug dialog. Example : incoming sms, notification, wifi connected/scanned event, cell id, location. You can also use it log something you have limited time of use in a certain period. Example : querying location info from google map service.

Example Usage
Some of the example usages of the flow (especially that logging script) :
1. Logging notification from certain app. As you can see in this flow example, I use it to log whatsapp notification. You can expand this to any app you want to log. So later you can check, that which variable contains the sender name/phone number, the message. You can then use it as filter or process the trigger based on it. Bonus tips : You can use this to read the deleted message before it get deleted. But you have to keep the notification clean and not stacked.
2. Logging wifi connected or scanned, bluetooth device connected, cell id, location and etc. You can check how many times you connected/disconnected to your wifi, so you can put your wifi in a better place. You can wardriving and collecting available wifi in a neighbour. Of course you should use it only for legal purpose, example as in geofencing. Wardriving itself should be legal in most cases, because it only logged the available network. But doing something illegal to the logged wifi AP, such as sniffing and bruteforcing the Wifi AP are illegal. Keep this in mind. Bluetooth device maybe not so needed in most cases. But cell id, location (along with wifi AP) can be useful for geofencing.
3. Logging UI event, to check for opened windows, checking if certain app contain certain text, button clicked, text changed and many other UI event related. After you know which one usually got triggered, you can use that as your trigger for the flow. This UI event also can be used to catch the app which produce random full screen ads. You can enable the flow and check when the ads pop up. Then go to the glovar and find out what is the app name. I use this in several occassion to find out which app is the culprit.
4. File or I/O event. You can detect the file/folder changes, moved, renamed, or any other file related operation. You can use file observer or init file list better when you know what is going on to the file/folder you need.
5. Clipboard logging. There are other better method to do it, but yeah you can do it using this flow.
6. Logging when you turn on and turn off you display, or when you turn on or shutdown the device. You can also use it to log how long you have been using a certain app. It can be useful when the app doesn't have timer. Example : I am learning some foreign language and most app doesn't have the time logging feature. I use automagic to log the time since I open the app until I close it, subtracting the value will give me the duration of study. Of course I have modified the script to suit my need.
7. Battery level logging. Of course, only log the value you need, don't log everything.
8. Logging App manager event, including the installation, update, uninstalled, and any changes. You can track any app event so you can modify your flow to follow the change. I used chrome custom user agent (set to Desktop). Everytime Chrome got updated, I have to find out the version, so I can modify the custom user agent version to match the new version number.
9. Logging for call, sms, email, and similar. Usually this will be used so we can act upon the event later. Example to filter certain sms and auto reply to the sender. Or to reject/auto answer certain call.
10. Toast message logging. Android doesn't provide us any detail info on which app producing the toast message. Using this logging, we can know that certain app produce certain toast message, so we can act on it. Example Mixplorer will toast file copied operation status. So we can check whether the operation success or failed.
11. Logging intent, general broadcast, web url intent. This is for advance usage. By logging the variables, it will helps us to determine what is the uri, extra, intent action and several other hidden variables; so we can fill them in the intent related action.

There are still so many things we can use the logging, the limit is your imagination.

Epilogue
Can't imagine it, a simple flow like this require so much time to document it here. But I can't stop typing after using this logging method for a long time. I actually want to share it after I finish the "Run with variables" from the logging result. But yeah, as you can see, it will take much more time and I will never complete it if I wait until that day come. Hopefully this flow will help you to find the variables you need, so you can create the flow you want. Happy logging. :)
Index of Automagic useful thread List of my other useful posts (and others')
Xiaomi Redmi Note 5 (whyred), Resurrection Remix Official 7.0.2 20190829, Pie 9.0, Rooted.

Post Reply