Android version: problem parsing XML with PHP

topics regarding problems or bugs in btraced

Android version: problem parsing XML with PHP

Postby ambrosa » Thu Apr 24, 2014 11:18 pm

Hi.

I'm a GPS tracking fan and I I've tried dozen apps in Google Play Store. I've not an iPhone: I don't like Apple devices.
My favourite app is "Logger GPS for Android" by mendhak. Opensource.

Yours app is good for 2 reason:
1) it use buffers for transmission
2) use OPEN protocol communication
3) Nice layout

The worst:
- DRAIN battery a lot. Probably the Android app with more drain that I've tried :-(
You should consider a 'low power' mode: switch-off GPS device between long acquisition ( >= 30 seconds) and in this low-power mode use BackgroundService like Logger GPS does (it can run for 1 days continously without draining battery).
The other apps that do it use something called WakeLock. WakeLock is basically a way of poking the CPU and asking it to stay awake for a little while longer. This drains battery so those apps for realtime tracking are meant for short trips or if they're on a constant power source.
Logger GPS isn't realtime and isn't meant to be, it's written so that it never asks for a WakeLock and instead implements BackgroundServices. In Android, BackgroundServices can be killed off when the OS decides. It can happen at anytime, so the logic involved also deals with the fact that the service might be dead and needs to come back for a little while. In other words, GPSLogger spends a lot of time sleeping and waiting.

- CANNOT save locally data into a GPX/XML file so I can later export it. No good. Sometimes I wish only to do local record without sending anything over the net (suppose you are for 7 days into mountains trekking/hiking without any 2G/3G/WiFi signal).
- It has no support for OpenGTS protocol (very simple : you can add into your app with a very small effort).


I'm developing a small and simple multiprotocol tracking website (OpenGTS, Custom URL, your XML format, ... other in future ....) using Debian 7.4 armhf + Lighttpd + PHP + Sqlite3 running in a small ARM board (better than Raspberry: it is Cubieboard.org).
For testing purpose I've bought your app and this afternoon I played with it in my Android phone (Sony Xperia T).
And always for testing purpose a friend of mine has bought the iPhone version (he has an iPhone 5C).

I used your "upload.php" as template and changed it to match my needs.
Well, my php code with iPhone xml strings works very fine. This is a debug of XML string received by my webserver :
Code: Select all
++ $BODY len:775
<bwiredtravel>
        <model>iPhone</model>
        <devId>FFFFFFFF26A99????????????4C5BE12F54</devId>
        <username>d???e</username>
        <password>s?????3</password>
        <timeOffset>7200</timeOffset>
        <travel>
                <id>1</id>
                <description>24/04/14%2016:55_trip</description>
                <length>24770.16</length>
                <time>5964</time>
                <tpoints>115</tpoints>
                <uplpoints>114</uplpoints>
                <point>
                        <id>140</id>
                        <date>1398365388.337045</date>
                        <lat>+4?.??0858</lat>
                        <lon>+9.????38</lon>
                        <speed>-1.000000</speed>
                        <course>-1.000000</course>
                        <haccu>65.000000</haccu>
                        <bat>0.50</bat>
                        <vaccu>10.000000</vaccu>
                        <altitude>132.764618</altitude>
                        <continous>1</continous>
                        <tdist>24770.16</tdist>
                        <rdist>0.28</rdist>
                        <ttime>5964</ttime>
                </point>
        </travel>
</bwiredtravel>
+++++ end BODY +++++


But with Android version the PHP XML parser doesn't work (libxml 2.8.0 PHP 5.4.4-14+deb7u8): simplexml_load_string() return 'false'.
This is an example string received by Android version:
Code: Select all
++ $BODY len:981
<?xml version='1.0' encoding='UTF-8' standalone='yes' ?><bwiredtravel><model>Android</model><devId>3538610538721131212</devId><username></username><password></password><timeOffset>7200</timeOffset><travel><id>1</id><getTripUrl>1</getTripUrl><description>Test 1</description><length>16.17</length><time>415</time><tpoints>6</tpoints><uplpoints>0</uplpoints><point><id>5</id><date>1398365292.556</date><lat>4?.???3357</lat><lon>9.????124</lon><speed>-1.0</speed><course>-1.00</course><haccu>40.435001373291016</haccu><bat>0.28</bat><vaccu>-1.0</vaccu><altitude>0.00</altitude><continous>1</continous><tdist>0.00</tdist><rdist>0.00</rdist><ttime>384</ttime></point><point><id>6</id><date>1398365323.000</date><lat>4?.????685</lat><lon>9.????467</lon><speed>-1.0</speed><course>-1.00</course><haccu>25.0</haccu><bat>0.28</bat><vaccu>-1.0</vaccu><altitude>175.10</altitude><continous>1</continous><tdist>16.17</tdist><rdist>6.30</rdist><ttime>415</ttime></point></travel></bwiredtravel>+++++ end BODY +++++

No info from libxml error reporting. Very strange.
After many test I found that the problem looks to be the "\n" missing in every line.
If I reformat the code above adding a "\n" for every line , now XML code is parsed correctly by PHP (now it looks to be as iPhone xml):
Code: Select all
<?xml version='1.0' encoding='UTF-8' standalone='yes' ?>
<bwiredtravel>
   <model>Android</model>
   <devId>3538610538721131212</devId>
   <username></username>
   <password></password>
   <timeOffset>7200</timeOffset>
   <travel>
      <id>1</id>
      <getTripUrl>1</getTripUrl>
      <description>Test 1</description>
      <length>16.17</length>
      <time>415</time>
      <tpoints>6</tpoints>
      <uplpoints>0</uplpoints>
      <point>
         <id>5</id>
         <date>1398365292.556</date>
         <lat>4?.????357</lat>
         <lon>9.????124</lon>
         <speed>-1.0</speed>
         <course>-1.00</course>
         <haccu>40.435001373291016</haccu>
         <bat>0.28</bat>
         <vaccu>-1.0</vaccu>
         <altitude>0.00</altitude>
         <continous>1</continous>
         <tdist>0.00</tdist>
         <rdist>0.00</rdist>
         <ttime>384</ttime>
      </point>
      <point>
         <id>6</id>
         <date>1398365323.000</date>
         <lat>4?.????685</lat>
         <lon>9.????467</lon>
         <speed>-1.0</speed>
         <course>-1.00</course>
         <haccu>25.0</haccu>
         <bat>0.28</bat>
         <vaccu>-1.0</vaccu>
         <altitude>175.10</altitude>
         <continous>1</continous>
         <tdist>16.17</tdist>
         <rdist>6.30</rdist>
         <ttime>415</ttime>
      </point>
   </travel>
</bwiredtravel>


2) <username> and <password> are ALWAYS empty but I've setup correctly your "Basic Auth". This is an error.

3) The iPhone version misses "<?xml version='1.0' encoding='UTF-8' standalone='yes' ?>" string (but it is not a real problem).

If you need further tests or you have some suggestion let me know.
Best Regards
---
Alex aka "ambrosa"
ambrosa
 
Posts: 5
Joined: Thu Apr 24, 2014 6:51 pm

Re: Android version: problem parsing XML with PHP

Postby ambrosa » Sun Apr 27, 2014 1:39 pm

I've solved issue (2) : was a my misunderstanding. <username> and <password> are OK.

I've rewritten all PHP code with XMLReader but performance are poor compared to SimpleXML.

I've done many tests and SimpleXML *REQUIRE* a well-formed XML

http://www.php.net/manual/en/function.s ... string.php :
data : A well-formed XML string
ambrosa
 
Posts: 5
Joined: Thu Apr 24, 2014 6:51 pm

Re: Android version: problem parsing XML with PHP

Postby smfunder » Mon Apr 28, 2014 10:11 pm

Hi Ambrosa,

Thank you for your comments, we really appreciate all the feedback provided. As far as I understand you figured out what was the problem loading the XML. So do you think it was just the new line break? If so we would be adding that on a future update on Android version.

It is strange SimpleXML cannot load an XML file without new line breaks.

We also appreciate the comment about the pro and issues of Btraced. We will try to enhance Btraced to make it the best application on GPS tracking. We already understand this consumes lots of battery, so our next focus will be fixing that as much as possible. We may lost some features when power-saving mode is on, but I hope this cover all the users requirements.

Thank you!
Santiago
smfunder
Site Admin
 
Posts: 17
Joined: Fri Mar 16, 2012 2:24 am

Re: Android version: problem parsing XML with PHP

Postby ambrosa » Tue Apr 29, 2014 7:06 am

The problem is that XML generated by Android version is NOT WELL-FORMED.
SimpleXML routine accept only "well-formed" XML document: this behaviour is described into SimpleXML documentation.

If I regenerate the XML with
xmllint --format XML_data
all works fine.

The solution is to output/send to server a well-formed xml document like Btraced iPhone version does.
Yes, with newline after every couple (or single) of tags:

<tagA>
<tagB>data></tagB>
<tagC>
<tagD><data></tagD>
</tagC>
</tagA>
ambrosa
 
Posts: 5
Joined: Thu Apr 24, 2014 6:51 pm

Re: Android version: problem parsing XML with PHP

Postby ambrosa » Tue Apr 29, 2014 9:32 pm

Ok solved the problem with some 'dirty' code: I preprocess XML with xmllint within PHP.
Now code for Android version works fine.
With my small ARM board (CubieBoard.org model 1) I've an overhead about 40 mSec for using xmllint. No good but it works ;)


This is the code:
Code: Select all
// Get the received data from the iPhone (XML data)

$xml_malformed = @file_get_contents('php://input');


$cmd = '/usr/bin/xmllint --format -';

$descriptorspec = array(
   0 => array("pipe", "r"), 
   1 => array("pipe", "w"), 
   2 => array('pipe', 'a')   
);

$process = proc_open($cmd, $descriptorspec, $pipes);

if (is_resource($process)) {
    // $pipes now looks like this:
    // 0 => writeable handle connected to child stdin
    // 1 => readable handle connected to child stdout

    fwrite($pipes[0], $xml_malformed);
    fclose($pipes[0]);

    $body = stream_get_contents($pipes[1]);
    fclose($pipes[1]);

    // It is important that you close any pipes before calling
    // proc_close in order to avoid a deadlock
    $return_value = proc_close($process);

}
else {
   print '{ "id":906, "error":true, "message":"Cant run XMLLINT", "valid":true }';
   exit();
}


// Try to load the XML
$xml = simplexml_load_string($body);

// continue with usual code.......
ambrosa
 
Posts: 5
Joined: Thu Apr 24, 2014 6:51 pm

Re: Android version: problem parsing XML with PHP

Postby silvones » Wed Jul 29, 2015 12:13 am

Hi,

I've rewritten all PHP code with XMLReader but performance are poor compared to SimpleXML.


Could you please share the PHP code with XMLReader?

@smfunder: Please fix the XML formatting issue on Android.

Thx
silvones
 
Posts: 1
Joined: Tue Jul 28, 2015 11:52 pm


Return to Report Problems or bugs

Who is online

Users browsing this forum: No registered users and 0 guests

cron