If you delete a message from your Inbox or one of your custom folders, it will still appear in [Gmail]/All Mail.

Here’s why: in most folders, deleting a message simply removes that folder’s label from the message, including the label identifying the message as being in your Inbox.

[Gmail]/All Mail shows all of your messages, whether or not they have labels attached to them.

If you want to permanently delete a message from all folders:

1. Move it to the [Gmail]/Trash folder (or [Gmail]/Bin or national version).
2. Delete it from the [Gmail]/Trash folder (or [Gmail]/Bin or national version).

All emails in [Gmail]/Spam and [Gmail]/Trash are deleted after 30 days.
If you delete a message from [Gmail]/Spam or [Gmail]/Trash, it will be deleted permanently.

Here’s how this looks like using Mail.dll .NET IMAP component:

// C# version:

using(Imap imap = new Imap())
{
	imap.ConnectSSL("imap.gmail.com");
	imap.UseBestLogin("user@gmail.com", "password");

	// Recognize Trash folder
	List<FolderInfo> folders = imap.GetFolders();
 	CommonFolders common = new CommonFolders(folders);
 	FolderInfo trash = common.Trash;

	// Find all emails we want to delete
	imap.SelectInbox();
	List<long> uids = imap.Search(Expression.Subject("email to delete"));

	// Move email to Trash
	List<long> uidsInTrash = new List<long>();
	foreach (long uid in uids)
	{
		long uidInTrash = (long)imap.MoveByUID(uid, trash);
		uidsInTrash.Add(uidInTrash);
	}

	// Delete moved emails from Trash
	imap.Select(trash);
	foreach (long uid in uidsInTrash)
	{
		imap.DeleteMessageByUID(uid);
	}

	imap.Close();
}

' VB.NET version:

Using imap As New Imap()
	imap.ConnectSSL("imap.gmail.com")
	imap.UseBestLogin("user@gmail.com", "password")

	' Recognize Trash folder
	Dim folders As List(Or FolderInfo) = imap.GetFolders()
	Dim common As CommonFolders = new CommonFolders(folders)
	Dim trash As FolderInfo = common.Trash

	' Find all emails we want to delete
	imap.SelectInbox()
	Dim uids As List(Of Long) = imap.Search(_
		Expression.Subject("email to delete"))

	' Move email to Trash
	Dim uidsInTrash As New List(Of Long)
	For Each uid As Long In uids
		Dim uidInTrash As Long = imap.MoveByUID(uid, trash)
		uidsInTrash.Add(uidInTrash)
	Next

	' Delete moved emails from Trash
	imap.[Select](trash)
	For Each uid As Long In uidsInTrash
		imap.DeleteMessageByUID(uid)
	Next

	imap.Close()
End Using

You can also use bulk methods to handle large amount of emails faster:

// C# version:

using(Imap imap = new Imap())
{
	imap.ConnectSSL("imap.gmail.com");
	imap.UseBestLogin("user@gmail.com", "password");

	// Recognize Trash folder
	List<FolderInfo> folders = imap.GetFolders();
 	CommonFolders common = new CommonFolders(folders);
 	FolderInfo trash = common.Trash;

	// Find all emails we want to delete
	imap.SelectInbox();
	List<long> uids = imap.Search(Expression.Subject("email to delete"));

	// Move emails to Trash
	List<long> uidsInTrash = imap.MoveByUID(uids, trash);

	// Delete moved emails from Trash
	imap.Select(trash);
	imap.DeleteMessageByUID(uidsInTrash);

	imap.Close();
}

' VB.NET version:

Using imap As New Imap()
	imap.ConnectSSL("imap.gmail.com")
	imap.UseBestLogin("user@gmail.com", "password")

	' Recognize Trash folder
	Dim folders As List(Or FolderInfo) = imap.GetFolders()
	Dim common As CommonFolders = new CommonFolders(folders)
	Dim trash As FolderInfo = common.Trash

	' Find all emails we want to delete
	imap.SelectInbox()
	Dim uids As List(Of Long) = imap.Search(Expression.Subject("email to delete"))

	' Move email to Trash
	Dim uidsInTrash As List(Of Long) = imap.MoveByUID(uids, trash)

	' Delete moved emails from Trash
	imap.[Select](trash)
	imap.DeleteMessageByUID(uidsInTrash)

	imap.Close()
End Using

Please also note that there are 2 additional sections in the “Gmail settings”/”Forwarding and POP/IMAP” tab:

“When I mark a message in IMAP as deleted:”

• Auto-Expunge on – Immediately update the server. (default)
• Expunge off – Wait for the client to update the server.

“When a message is marked as deleted and expunged from the last visible IMAP folder:”

• Archive the message (default)
• Move the message to the Trash
• Immediately delete the message forever

Those settings change the default behavior of the account and modify DeleteMessage* methods behavior.

In this article I’ll show how to access Gmail account of any domain user, using OAuth 2.0, .NET IMAP component and service accounts. The basic idea is that domain administrator can use this method to access user email without knowing user’s password.

This scenario is very similar to 2-legged OAuth, which uses OAuth 1.0a. Although it still works, it has been deprecated by Google and OAuth 2.0 service accounts were introduced.

The following describes how to use XOAUTH2 and OAuth 2.0 to achieve the equivalent of 2-legged OAuth.

Google.Apis

Use Nuget to download “Google.Apis.Auth” package.

Import namespaces:

// c#

using Google.Apis.Auth.OAuth2;
using Google.Apis.Auth.OAuth2.Flows;
using Google.Apis.Auth.OAuth2.Requests;
using Google.Apis.Auth.OAuth2.Responses;

using System.Security.Cryptography.X509Certificates;

using Limilabs.Client.Authentication.Google;

using Limilabs.Client.IMAP;

' VB.NET 

Imports Google.Apis.Auth.OAuth2
Imports Google.Apis.Auth.OAuth2.Flows
Imports Google.Apis.Auth.OAuth2.Requests
Imports Google.Apis.Auth.OAuth2.Responses

Imports System.Security.Cryptography.X509Certificates

Imports Limilabs.Client.Authentication.Google

Imports Limilabs.Client.IMAP

Google APIs console

First you need to visit the Google Developers Console and create a service account:

Download and save this private key (XYZ.p12), you’ll need that later:

Go to “Service accounts” and click “View Client ID”:

Make a note of the Client ID and Email address (Service account).

Google Apps Dashboard

Next step is to authorize access for newly created service account.

Visit your domain administration panel:
https://www.google.com/a/cpanel/yourdomain.com/ManageOauthClients

Then click “Advanced tools”, “Authentication” and “Manage third party OAuth Client access”.

On this screen you can authorize service account to access email scope:

Use previously remembered Client ID and “https://mail.google.com/”, which is IMAP/SMTP API scope:

Client Name: 1234567890
One or More API Scopes: https://mail.google.com/

Access IMAP/SMTP server

// C#

const string serviceAccountEmail = "name@xxxxxxxxxx.gserviceaccount.com";
const string serviceAccountCertPath = @"c:\XYZ.p12";
const string serviceAccountCertPassword = "notasecret";
const string userEmail = "user@your-domain.com";

X509Certificate2 certificate = new X509Certificate2(
    serviceAccountCertPath, 
    serviceAccountCertPassword, 
    X509KeyStorageFlags.Exportable);

ServiceAccountCredential credential = new ServiceAccountCredential(
    new ServiceAccountCredential.Initializer(serviceAccountEmail)
    {
        Scopes = new[] { GoogleScope.ImapAndSmtp.Name },
        User = userEmail
    }.FromCertificate(certificate));


bool success = await credential.RequestAccessTokenAsync(CancellationToken.None);

using (Imap imap = new Imap())
{
    imap.ConnectSSL("imap.gmail.com");
    imap.LoginOAUTH2(userEmail, credential.Token.AccessToken);

    imap.SelectInbox();

    foreach (long uid in uids)
    {
        var eml = client.GetMessageByUID(uid);
        IMail email = new MailBuilder().CreateFromEml(eml);
        Console.WriteLine(email.Subject);
    }

    imap.Close();
}

' VB.NET

Const serviceAccountEmail As String = "name@xxxxxxxxxx.gserviceaccount.com"
Const serviceAccountCertPath As String = "c:\XYZ.p12"
Const serviceAccountCertPassword As String = "notasecret"
Const userEmail As String = "user@your-domain.com"

Dim certificate As New X509Certificate2(serviceAccountCertPath, serviceAccountCertPassword, X509KeyStorageFlags.Exportable)

Dim credential As New ServiceAccountCredential(New ServiceAccountCredential.Initializer(serviceAccountEmail) With { _
	.Scopes = {GoogleScope.ImapAndSmtp.Name}, _
	.User = userEmail _
}.FromCertificate(certificate))

Dim success As Boolean = credential.RequestAccessTokenAsync(CancellationToken.None).Result

Using imap As New Imap()
	imap.ConnectSSL("imap.gmail.com")
	imap.LoginOAUTH2(userEmail, credential.Token.AccessToken)

	imap.SelectInbox()

	For Each uid As Long In uids
		Dim eml = client.GetMessageByUID(uid)
		Dim email As IMail = New MailBuilder().CreateFromEml(eml)
		Console.WriteLine(email.Subject)
	Next

	imap.Close()
End Using

IMAP protocol uses Polish notation (also known as prefix notation) for client queries. In general it means, that operator (in case of IMAP search it’s always a logical operator) is placed left of its operands.

Polish notation

Expression that would be written in conventional infix notation as:
(5 − 6) * 7
can be written in prefix/Polish notation as:
* (− 5 6) 7
As all mathematical operands are binary (or have defined number of operands) we don’t need parentheses to correctly evaluate this expression:
* − 5 6 7

Polish notation doesn’t require parentheses.

IMAP search needs parentheses

Although IMAP search syntax uses Polish notation it actually requires parentheses to be evaluated unambiguously.

This is because there is no explicitly defined AND operator, that takes exactly 2 operands.

RFC 3501:

When multiple keys are specified, the result is the intersection (AND function) of all the messages that match those keys.

So although it is true that Polish and reverse Polish notations don’t require parentheses, it is not true that IMAP search doesn’t require them.

‘OR/AND’ example

Expression 1a:

Regular notation:
(SUBJECT subject OR BODY body) AND FROM from

Polish notation:
AND OR SUBJECT subject BODY body FROM from

IMAP’s Polish notation (note there is no AND operator):
OR SUBJECT subject BODY body FROM from

Expression 1b:

Regular notation:
SUBJECT subject OR (BODY body AND FROM from)

Polish notation:
OR SUBJECT subject AND BODY body FROM from

IMAP’s Polish notation (note there is no AND operator):
OR SUBJECT subject BODY body FROM from

1a != 1b

Expression 1a is not equal to Expression 1b:
((a or b) and c) != (a or (b and c))

For:
a = true
b = true
c = false

We have:
((true or true) and false) != (true or (true and false))
false != true

Parentheses are needed

But when there is no AND operator, IMAP’s Polish notations look exactly the same, thus we need parentheses!

Expression 1a (IMAP’s Polish notation with parentheses):
(OR SUBJECT subject BODY body) FROM from

Expression 1b (IMAP’s Polish notation with parentheses):
OR SUBJECT subject (BODY body FROM from)

OR SUBJECT subject BODY body FROM from
is treated like
(OR SUBJECT subject BODY body) FROM from

‘NOT’ example

Expression 2a:

Regular notation:
NOT (SUBJECT subject AND BODY body)

Polish notation:
NOT AND SUBJECT subject BODY body

IMAP’s Polish notation (note there is no AND operator):
NOT SUBJECT subject BODY body

Expression 2b:

Regular notation:
(NOT SUBJECT subject) AND BODY body

Polish notation:
AND NOT SUBJECT subject BODY body

IMAP’s Polish notation (note there is no AND operator):
NOT SUBJECT subject BODY body

2a != 2b

Expression 2a is obviously not equal to Expression 2b.

Parentheses are needed

Again IMAP’s Polish notations look the same. We need parentheses!

Expression 2a (IMAP’s Polish notation with parentheses):
NOT (SUBJECT subject BODY body)

Expression 2b (IMAP’s Polish notation with parentheses):
(NOT SUBJECT subject) BODY body

NOT SUBJECT subject BODY body
is treated like
(NOT SUBJECT subject) BODY body

IMAP protocol in RFC3501 introduced \Answered flag. \Answered flag should mark messages that have been answered. This flag can by set by client applications or even by SMTP server, which can be examining In-Reply-To email headers.

Unfortunately if the message is answered through Gmail’s web interface, the \Answered flag will not be set. It will only be set if the messages were answered using an email program that sets this flag.

This means that \Answered flag can’t be used to find not-answered emails.

One option to resolve this problem is to use Gmail IMAP extensions, X-GM-THRID in particular.

We’ll connect to Gmail using IMAP – remember to use SSL and be sure to enable IMAP protocol for Gmail.

As we plan to search all emails, including sent and received ones, we’ll use CommonFolders class to get ‘All Mail’ folder.

Then we’ll get Envelope for each message. Envelope contains GmailThreadId property. It contains thread id to which this email was assigned to by Gmail.

Finally we’ll find threads that contain only one message – those are messages that were not answered.

// C#

using(Imap client = new Imap())
{
    client.ConnectSSL("imap.gmail.com");
    client.Login("pat@limilabs.com","password");

    // Select 'All Mail' folder.
    List<FolderInfo> folders = client.GetFolders();
    CommonFolders common = new CommonFolders(folders);
    client.Select(common.AllMail);

    // Get envelopes for all emails.
    List<long> uids = client.GetAll();
    List<Envelope> envelopes = client.GetEnvelopeByUID(uids);

    // Group messages by thread id.
    var threads = new Dictionary<decimal, List<Envelope>>();
    foreach (Envelope envelope in envelopes)
    {
        decimal threadId = (decimal) envelope.GmailThreadId;
        if (threads.ContainsKey(threadId) == false)
            threads[threadId] = new List<Envelope>();
        
        threads[threadId].Add(envelope);
    }

    // Find threads containing single message.
    foreach (KeyValuePair<decimal, List<Envelope>> pair in threads)
    {
        if (pair.Value.Count == 1)
        {
            Envelope envelope = pair.Value[0];
            Console.WriteLine("Gmail message id: {0}; subject: {1}", 
                envelope.GmailMessageId , 
                envelope.Subject);

        }
    }
    client.Close();
}

' VB.NET

Using client As New Imap()
	client.ConnectSSL("imap.gmail.com")
	client.Login("pat@limilabs.com", "password")

	' Select 'All Mail' folder.
	Dim folders As List(Of FolderInfo) = client.GetFolders()
	Dim common As New CommonFolders(folders)
	client.[Select](common.AllMail)

	' Get envelopes for all emails.
	Dim uids As List(Of Long) = client.GetAll()
	Dim envelopes As List(Of Envelope) = client.GetEnvelopeByUID(uids)

	' Group messages by thread id.
	Dim threads = New Dictionary(Of Decimal, List(Of Envelope))()
	For Each envelope As Envelope In envelopes
		Dim threadId As Decimal = CDec(envelope.GmailThreadId)
		If threads.ContainsKey(threadId) = False Then
			threads(threadId) = New List(Of Envelope)()
		End If

		threads(threadId).Add(envelope)
	Next

	' Find threads containing single message.
	For Each pair As KeyValuePair(Of Decimal, List(Of Envelope)) In threads
		If pair.Value.Count = 1 Then
			Dim envelope As Envelope = pair.Value(0)

			Console.WriteLine("Gmail message id: {0}; subject: {1}", _
				envelope.GmailMessageId, envelope.Subject)
		End If
	Next
	client.Close()
End Using

Attachments are not stored separately from message text and headers – they are embedded inside an email message. This, along with inefficient Base64 encoding is the most important reason of email messages being large in size. Mail.dll provides an easy way to replace attachments in existing messages:

// C#

IMail email = new MailBuilder().CreateFromEml(eml);
email.ReplaceAttachments();

' VB.NET

Dim email As IMail = New MailBuilder().CreateFromEml(eml)
email.ReplaceAttachments()

Each attachment will be replaced with the following text information: “This file (‘[FileName]’) containing [Size] bytes of data was removed.”. Thus making email much smaller in size.

ReplaceAttachmentsmethod has an overloaded version, that allows you to skip visual elements (content-disposition: inline) or/and alternative email representations. It also allows to specify text template and custom Tag, that can be used, for example, to create a custom url. This url can point to a place to which attachment was moved.

Within the template you can use [FileName], [Size] and [Tag] as template placeholders.

// C#

IMail email = Limilabs.Mail.Fluent.Mail
    .Text("body")
    .AddAttachment(new byte[] { 1, 2, 3 })
    .SetFileName("report.pdf")
    .Create();

AttachmentReplacerConfiguration configuration = new AttachmentReplacerConfiguration();
configuration.ReplaceVisuals = false;
configuration.Tag = 
    att => "http://example.com/" + email.MessageID + "/" + att.FileName;
configuration.Template = 
    "Attachment [FileName] removed. You can download it here: [Tag]";

email.ReplaceAttachments(configuration);

' VB.NET

Dim email As IMail = Limilabs.Mail.Fluent.Mail _
    .Text("body") _
    .AddAttachment(New Byte() {1, 2, 3}) _
    .SetFileName("report.pdf") _
    .Create()

Dim configuration As New AttachmentReplacerConfiguration()
configuration.ReplaceVisuals = False
configuration.Tag = Function(att)
    Return "http://example.com/" + email.MessageID + "/" + att.FileName
    End Function
configuration.Template = _
    "Attachment [FileName] removed. You can download it here: [Tag]"

email.ReplaceAttachments(configuration)

The following example illustrates the full process of downloading email from IMAP server,
creating new email, with the same information, but with all attachments replaced, uploading this message, and deleting original one:

// C#

using(Imap imap = new Imap())
{
    imap.ConnectSSL("imap.example.org");
    imap.UseBestLogin("user", "password");
    imap.SelectInbox();

    foreach (long uid in imap.GetAll())
    {
        var eml = imap.GetMessageByUID(uid);
        IMail email = new MailBuilder().CreateFromEml(eml);
        if (email.Attachments.Count > 0)
        {
            email.ReplaceAttachments();

            imap.UploadMessage(email);

            imap.DeleteMessageByUID(uid);
        }
    }
    imap.Close();
}

' VB.NET

Using imap As New Imap()
   imap.ConnectSSL("imap.example.org")
   imap.UseBestLogin("user", "password")
   imap.SelectInbox()

   For Each uid As Long In imap.GetAll()
      Dim eml = imap.GetMessageByUID(uid)
      Dim email As IMail = New MailBuilder().CreateFromEml(eml)
      If email.Attachments.Count > 0 Then
         email.ReplaceAttachments()

         imap.UploadMessage(email)

         imap.DeleteMessageByUID(uid)
      End If
   Next
   imap.Close()
End Using