working with parser's output

thanks. I have tried to connect to my mail server and kept on not being able to open the message so I thought maybe its the settings of the host or something. SO I tried the GMAIL account using:

$message_file='pop3://'.$GMAILusername.':'.$GMAILpassword.'@pop.gmail.com:995/1?tls=1&debug=1&html_debug=1';


I then still wasnt able to download the message. And I keep getting the following error so it was perhaps correct in the first way I was accessing it with my mail account.


MIME message decoding error: could not open the message file to decode pop3://xxxusername:[email protected]:995/1?tls=1&debug=1&html_debug=1 at position -1

There is 1 message in that gmail account. So im not sure what the problem could be that the following is set: $mime->mbox = 1; ?

What could be causing the problem of not being able to open the message?

Kind regards
Angelo

I think you are not able to connect to Gmail POP3 server at all. It is possible that your server may have blocked outgoing port 995 in the firewall.

If you look at the debug output and see that class attempts to connect to the server but it does not pass from that point, your connection may be blocked your network firewall. Ask your system administrator about that.

I dont think its the problem. I get the same error message when I try to use a different mail account on a different server. As you can see from the code below I can connect to the mailbox.

Please let me know if I am doing this correctly:

Connect to POP3 server: $pop3->Open()
Get number of messages: $pop3->ListMessages()
Loop through messages: for($message=0;$message<count($result);$message++)
Try parse each message using mime parser class


This is the output
//=========================//
Connected to the POP3 server "mail.xx.co.za".

User "[email protected]" logged in.

Message 1 - 1438672 bytes.

Message 1, Unique ID - "1171881746.H827005P15424.dedi53b.your-server.co.za"


messages: 1
MIME message decoding error: could not open the message file to decode pop3://xx:xxmail.xx.co.za:110/1?debug=1&html_debug=1&realm=&workstation=&apop=0&authentication_mechanism=USER at position -1

Disconnected from the POP3 server "mail.xx.co.za".
//=========================//

So Im assuming that there is no network/firewall problem. What other issues could there be that are not allowing the decoding of the message? because you can see that we can connect to the mailbox and see all the messages and loop through them. Could it be some of the arguments that are used when trying to decode the message using POP:// ... ?

Anything else that you think I can try?


This is the complete code:

require("pop3.php");

/* Uncomment when using SASL authentication mechanisms */
/*
require("sasl.php");
*/

$pop3=new pop3_class;
$pop3->hostname="mail.xxx.co.za"; /* POP 3 server host name */
$pop3->port=110; /* POP 3 server host port,*/


//------GMAIL-----
// $pop3->hostname="pop.gmail.com"; /* POP 3 server host name */
// $pop3->port=995;
/* POP 3 server host port,

usually 110 but some servers use other ports
Gmail uses 995 */

$pop3->tls=0; /* Establish secure connections using TLS */
// $GMAILusername="x";
$username="x";
$user="[email protected]"; /* Authentication user name */
// $GMAILpassword="xxx"; /* Authentication password */
$password="xx"; /* Authentication password */
$realm=""; /* Authentication realm or domain */
$workstation=""; /* Workstation for NTLM authentication */
$apop=0; /* Use APOP authentication */
$authentication_mechanism="USER"; /* SASL authentication mechanism */
$debug=1; /* Output debug information */
$html_debug=1; /* Debug information is in HTML */
$join_continuation_header_lines=1; /* Concatenate headers split in multiple lines */

$msg="";
$end_of_message="";

if(($error=$pop3->Open())=="")
{
echo "<PRE>Connected to the POP3 server &quot;".$pop3->hostname."&quot;.</PRE>\n";
if(($error=$pop3->Login($user,$password,$apop))=="")
{
echo "<PRE>User &quot;$user&quot; logged in.</PRE>\n";
if(($error=$pop3->Statistics($messages,$size))=="")
{
////echo "<PRE>There are $messages messages in the mail box with a total of $size bytes.</PRE>\n";
$result=$pop3->ListMessages("",0);
if(GetType($result)=="array")
{
for(Reset($result),$message=0;$message<count($result);Next($result),$message++)
echo "<PRE>Message ",Key($result)," - ",$result[Key($result)]," bytes.</PRE>\n";

$result=$pop3->ListMessages("",1);
if(GetType($result)=="array")
{
echo "<br><br>";
for(Reset($result),$message=0;$message<count($result);Next($result),$message++)
echo "<PRE>Message ",Key($result),", Unique ID - \"",$result[Key($result)],"\"</PRE>\n";

echo "<br>messages: " . $messages . " <br>";

if($messages>0)
{

for($message=0;$message<count($result);$message++)
{



// $pop3->OpenMessage($message+1, -1);
// $pop3->GetMessage(1438672, $msg, $end_of_message);


require_once('mimeparser/mime_parser.php');
$mime=new mime_parser_class;
/*
* Set to 0 for parsing a single message file
* Set to 1 for parsing multiple messages in a single file in the mbox format
*/
$mime->mbox = 0;

/*
* Set to 0 for not decoding the message bodies
*/
$mime->decode_bodies = 1;

/*
* Set to 0 to make syntax errors make the decoding fail
*/
$mime->ignore_syntax_errors = 1;


//Get the file using the POP URL....
$message_file='pop3://'.$username.':'.$password.'@mail.xxxx.co.za:110/'.($message+1).
'?debug='.$debug.'&html_debug='.$html_debug.'&realm='.$realm.'&workstation='.$workstation.
'&apop='.$apop.'&authentication_mechanism='.$authentication_mechanism;


//$message_file='pop3://'.$GMAILusername.':'.$GMAILpassword.'@pop.gmail.com:995/1?tls=1&debug=1&html_debug=1';

$parameters=array(
'File'=>$message_file,

/* Save the message body parts to a directory */
'SaveBody'=>'extracted/',

/* Do not retrieve or save message body parts */
'SkipBody'=>1,
);

if(!$mime->Decode($parameters, $decoded))
echo 'MIME message decoding error: '.$mime->error.' at position '.$mime->error_position."\n";
else
{
echo 'MIME message decoding successful.'."\n";
echo (count($decoded)==1 ? '1 message was found.' : count($decoded).' messages were found.'),"\n";
for($message = 0; $message < count($decoded); $message++)
{
echo 'Message ',($message+1),':',"\n";
var_dump($decoded[$message]);
}
for($warning = 0, Reset($mime->warnings); $warning < count($mime->warnings); Next($mime->warnings), $warning++)
{
$w = Key($mime->warnings);
echo 'Warning: ', $mime->warnings[$w], ' at position ', $w, "\n";
}
}

}//end for loop
}
if($error==""
&& ($error=$pop3->Close())=="")
echo "<PRE>Disconnected from the POP3 server &quot;".$pop3->hostname."&quot;.</PRE>\n";

}
else
$error=$result;
}
else
$error=$result;
}
}
}
if($error!="")
echo "<H2>Error: ",HtmlSpecialChars($error),"</H2>";

Ok I think that you are correct, because I see in your class you use fopen to open that message. I have tried many variations of the pop3://...

but cant get it to work. Are you sure you can open the message like that with fopen? I see in your example you use a file in the file system as an example file which works:

$message_file=( (IsSet($_SERVER['argv']) && count($_SERVER['argv'])>1) ? $_SERVER['argv'][1] : 'test/sample/message.eml');

I have been trying to research if that way is possible. I also tried with the gmail method and fopen just using my username and password. Didnt work either.

Is it not possible for you to give an example of using the pop3:// way as the $message_file parameter?

Thanks again.

You need to be running at least PHP 4.3 and register the POP3 stream wrapper like in the parse_message.php example:

stream_wrapper_register('pop3', 'pop3_stream');

Note that you cannot open two connections to the same POP3 mailbox. Either you close the POP3 connection before using the MIME parser class to parse the messages as a file, or you use OpenMessage and GetMessage functions to get the messages to a string.

In that later case you need pass the message data string in the Decode function of the MIME parser class with the Data parameter and do not set the File parameter at all.

HI Manuel.

Thanks I got it working seperately as the previous reply seemed to help me so I connected just once to the mailbox and success! Thank-you!

Manuel,

First of all thank you for taking the time to write such useful classes and share them with the PHP community, as well as provide time to do tech support as well.

I think I have a situation similar to many people here. I want to loop through all the messages in a POP mailbox, decode all the attachments, and delete them when done. Your classes definitely allow one to do that but it seems like a lot of work to do this in a flexible way (i.e. handling large messages, etc). I can think of two solutions:

1) Using the pop3 stream wrapper you don't have to worry about message size. However you can't delete the messages either, which is a pain. And you don't even know how many messages there are, so you don't know what message ids you can legally put in the pop stream URL.

So first you'd have to open a non stream connection to get the message count. Then you can loop through those messages using a pop3 stream and decode the attachments. But in order to know what messages you've processed so that you can delete them later, as you're doing the mime decoding, you'd have to cache the message IDs in a database table, then close the pop3 stream and re-open the POP mailbox without the stream, iterate through all the messages, and any messages with IDs in the database, delete them.

Seems like a lot of work and I don't like the idea of using a database - seems to make things needlessly complicated - but it seems there is no other way to keep track of which messages you have processed so that you don't accidentally delete new messages that might have come in immediately after you looped through all the messages using a pop3 stream.

2) Using the pop3 class you can write each message to a disk file and delete them at the same time. Then you can pass each file to the mime parser class. You yourself listed several negatives to this method including file permissions, disk space, etc.

Are there any easier ways? Surely this must be a common problem... I guess it seems like #2 is easiest. The pop3 stream wrapper is a nice idea but pretty impractical if it doesn't allow iterating through all messages and doesn't allow deleting messages in my opinion. (I know you said you were thinking of improving it??)

thanks,
-Jim

You are right. The problem is that I have not figured an elegant solution that I can implement without taking me a lot of time.

Ideally you would reuse an opened POP3 connection. The problem is how can I pass an identifier of an object via the URL passed to fopen?

An alternative solution is to provide means to pass chunks of data read from the POP3 class with GetMessage. For that I need to redo a bit the MIME parser class.

I need more time to think about which is the best solution.

If somebody needs to parse ALL the parts of an email and get one array with every part in it, use this recursive function:

function arrayFromEmailParts($parts,&$index){
foreach ($parts as $part) {
$tmp = explode(';',$part['Headers']['content-type:']);
$type = $tmp[0];
// multipart/alternative
if ($type == 'multipart/alternative') {
$ret = array_merge($ret,arrayFromEmailParts($part['Parts'],$index));
} else {
if ($type == 'text/plain' OR $type == 'text/html') {
$ret[$index]['type'] = $type;
$ret[$index]['content'] = file_get_contents($part['BodyFile']);
} else {
$ret[$index]['type'] = $type;
$ret[$index]['file'] = $part['BodyFile'];
foreach($tmp as $data) {
if (strstr($data,'name')) {
$t = explode("=",$data);
$fileName = $t[1];
// outlook (and maybe others?) puts "" around the name, so remove if found
if(substr($fileName,0,1) == '"' AND substr($fileName,strlen($fileName)-1,1) == '"')
$fileName = substr($fileName,1,strlen($fileName)-2);

$ret[$index]['originalFileName'] = $fileName;
}
}
}
$index++;
}
}
return($ret);
}


Use the function like this:

// parameters for the mime parser
$parameters = array(
'Data' => $email,
'SaveBody' => "tmp" // saves every part as a file in tmp directory
);
// Decode
$mime->Decode($parameters, $decoded);
$decoded = $decoded[0];$index = 0;
$parts = $decoded['Parts'];
$result = arrayFromEmailParts($parts,$index);
print_r($result);

It will return an array with every part of the email and a few information about that part ('content' if plain text or html or 'file' and 'originalFileName' if it's something else).

An example of the array you will get when you parse a gmail message with 3 files attached:

(
[0] => Array
(
[type] => text/plain
[content] => 3 files, a rtf, a pdf and a picture.
)

[1] => Array
(
[type] => text/html
[content] => 3 files, a <b>rtf</b>, a <b>pdf</b> and a <b>picture</b>.

)

[2] => Array
(
[type] => application/rtf
[file] => tmp/3
[originalFileName] => eMarketing.rtf
)

[3] => Array
(
[type] => application/pdf
[file] => tmp/4
[originalFileName] => 2.pdf
)

[4] => Array
(
[type] => image/jpeg
[file] => tmp/5
[originalFileName] => IMG_4538-net.jpg
)

)



Tested with mails from gmail, outlook and Mail App (from apple).

One intersting thing is that outlook has some "" around the name of the file while apple mail does not have that.

Anyway, hope it helps!