Downloading body of mail in gmail takes longer

+1 vote

Hi,
I have purchased a license 2nd july from limilabs, so expecting technical support for this.

Problem:
I'm using imap.GetMessageInfoByUID(uids) method to download headers part of mails from gmail and this is quite faster. But it does not have body of the mail. I need to fetch body but that needs an extra method BodyStructure structure = imap.GetBodyStructureByUID(uid) which is very slow as i'm looping it. Think about I have more than 4 lac of emails in my gmail account and it is taking too much time. Is there any solution can you provide so that fetching body will take less time.

Even I don't need whole body. I need only first 200 characters from the body. There seems no method like this in limilabs. Can you provide a method something like this so that we can fetch only 200 char and the speed will be too fast as compare to GetBodyStructureByUID method.

Waiting for your reply.

Thanks,
Sarathi

by (1.4k points)

1 Answer

0 votes
 
Best answer

1.
GetMessageInfoByUID downloads BodyStructure (MessageInfo.BodyStructure property) so you don't need to invoke GetBodyStructureByUID separately.

2.
When using GetMessageInfoByUID use overload that takes list of uids as a parameter (Imap.GetMessageInfoByUID(List uids)) as a parameter. It's certainly faster than looping over a uid list.

3.

Even I don't need whole body. I need only first 200 characters from the body.

This doesn't make much of difference in terms of speed.

4.
Use GetTextByUID to download text data:

string text = imap.GetTextByUID(info.BodyStructure.Text);

There is also a GetTextByUID bulk overload, that takes list of structures as a parameter: Imap.GetTextByUID(List structures). It fills every structure's Text and Data properties.

by (297k points)
selected by
Thanks for quick reply.
But GetMessageInfoByUID has a property called BodyStructure.
That keeps only Attachments names. The html and text part of BodyStructure is always null. So fetching body I have to use GetBodyStructureByUID method which holds body and this method is too slow in the loop of having 4 lacs of mail.
by (1.4k points)
Text and Html properties of BodyStructure are not always null. Don't loop - use bulk methods.
by (297k points)
info.BodyStructure.Text.text. Here is the only method to see the body. And it's always null. You can read this link:

https://www.limilabs.com/blog/get-email-information-from-imap-fast

So it does not hold body of email.
by (1.4k points)
Please read the article you mentioned and my answer carefully. BodyStructure.Text.Text is null unless you use GetTextByUID method to retrieve data for that specific MIME part. Use bulk methods for efficiency.
by (297k points)
So I'm using List<MessageInfo> infos = imap.GetMessageInfoByUID(uids) to get all header info.

Can you please help me how to use bulk method of imap.GetTextByUID. I mean it needs <List> MimeTextStructure as a parameter. How can i get that structure from List<MessageInfo> infos as it does not have property called bodystructure. If I loop it and keep only info object as MessageInfo info = infos[i]; then only each info object has that property.

Can you please help me in that.
by (1.4k points)
MessageInfo class has BodyStructure property (of BodyStructure type). BodyStructure has Text property (of MimeTextStructure type).
by (297k points)
That's the problem.
I have List<MessageInfo> infos (List of MessageInfo). So to get each MessageInfo I need to put it on loop. So how can i use bulk method of GetTextByUID as it needs List of MimeTextStructure.
by (1.4k points)
Are you asking how to create a list of objects using a loop? I'm not sure how can I help you with that: use a loop and create new list instance.
by (297k points)
edited by
Yes, That's wht i'm doing. Using loop and getting each MessageInfo and created a list of MimeTextStructure.

So conclusion is that anyhow I need to loop through List<MessageInfo> infos to use bulk method of GetTextByUID.
by (1.4k points)
There is a big difference between calling a server inside a loop multiple times and preparing a parameter for a single server call.
by (297k points)
Yes you are right.
I just tried with 16K mails.
While List<MessageInfo> infos took only 1 minutes, but GetTextByUID using bulk took 20 minutes.

So think, If we fetch more than 4 lac mails using GetTextByUID bulk method, it will take approx 500 min. (more than 8 hours). So is there anyway to fetch it faster. How outlook is doing so fast using same IMAP gmail server.
by (1.4k points)
1. I don't know how your code looks like. 2 Outlook doesn't download text for all emails. 3. There is no other way. 4. By downloading this amount of data you may be experiencing memory problems. 5. I don't understand what "lac" is?
by (297k points)
Here is my code:

            List<MessageInfo> infos = imap.GetMessageInfoByUID(uids);
            List<MimeTextStructure> struc=new List<MimeTextStructure>();          
            foreach (var item in infos)
            {
                if (item.BodyStructure.Text!=null)
                    struc.Add(item.BodyStructure.Text);
            }
            imap.GetTextByUID(struc);

So List<MessageInfo> infos = imap.GetMessageInfoByUID(uids); this takes only 1 minute.
But imap.GetTextByUID(struc); takes 20 minute. (16000 of mails).
by (1.4k points)
Are you sure that Gmail is not throttling you? Those numbers  are way to big. Text data for 1000 emails I get in 30 seconds.
by (297k points)
I don't think so. I tried with different Gmail account.

It may be issue with internet speed.

Anyway thank you very much for your time and input.
by (1.4k points)
I also noticed that getting new messages from Gmail is faster than retrieving old ones (~20% difference).
by (297k points)
I just tried switching over to the bulk method and I get random errors like this.

 imap.GetTextByUID(struc);

"Tried to read 30340 bytes, only 16348 received. Please make sure that antivirus and firewall software are disabled or configured correctly."

When I do it non bulk and just loop it, it always works, just takes a long time.

Say I have 16000 structures and I'm sure that it's failing on one of the 16000 in the bulk - Then I have to try to do all 16000 again. Is there a way - to suppress the error and have mail.dll just try to re-download the one it failed on, so I don't have to attempt all 16000 again in bulk.
by (101 points)
Unfortunately you can't suppress this error. This is a serious communication error, and usually means that either connection was cut or the server is acting incorrectly (it send less data than it previously declared). After this error is raised you need to reconnect.

You may try increasing timeout values: Imap.ReceiveTimeout and Imap.SendTimeout.

I'd also suggest using smaller bulks, this way you'll be retrying with much less data lost.
by (297k points)
Increasing the Timeouts worked, I no longer get the errors. Thanks.  I do agree with the other poster, even using the bulk method it does take a very long time. 16,000 emails does take about 20 minutes.
by (101 points)
...