CFPOP – deleting an email when there is a comma in the UID

It seems that a problem has existed with the CFPOP delete action for a number of versions and it has not been fixed. The problem occurs when trying to delete a single email (or a list of emails) from the mail server where the mail server has included a comma in the UID. eg “1382396111.12731.mydomain.net,S=965”

The CF docs for the CFPOP Delete action says: “UID or a comma-separated list of UIDs to get or delete. Invalid UIDs are ignored.”

So, the abovementioned UID gets split into 2 uids (notice that it has a comma in it) and of course the email is not located so cant be deleted. CFPOP does not include a DELIMITERS parameter to use with the UID paramater which would solve the problem, I think. eg

 
<cfset myUIDlist = "1382051456.15221.mydomain.net,S=965|1382051456.18000.mydomain.net,S=966" />  
<cfpop server = "********" username = "********" password = "******** 
	action = "Delete" uid="#myUIDlist#" delimiter="|" >

Fortunately, there is a solution – use POP CFC instead of CFPOP! Thanks to Paul Vernon.

So, here is a sample page to demonstrate a way to use POP CFC.

<cfscript>
/**
 * Example of reading, processing then deleting individual emails from the pop mail server
 * using the POP CFC custom component.
 * http://popcfc.riaforge.org/
 */

	// To make it easier to rescope this example (local, variables ... whatever)
variables.s = {};

			// Initialise the custom POP CFC component. Used instead of CFPOP
			// Set your values here
s.popAccount = createObject("component", "pop").init(myPopServerHostname, myUsername, myPassword)

			// Specify a non-comma separator for the UID lists. 
			// You must do this in order for the list functionality
			// of POP CFC to work (including deletions)
s.mySeparator = "|";
s.popAccount.setProperty("separator",s.mySeparator);

			// Get an array of the UIDs of the messages in the mail account.
			// ie retrieve them from the mail server
			// This is a necessary step to solve the comma-in-uid problem
s.UIDArray = s.popAccount.getUIDList();

if (arrayLen(s.UIDArray)) {
			// Convert the array to a list
			// Keep the list to use for deletions
	s.UIDList = ArrayToList(s.UIDArray, s.mySeparator);

			// Get an array of the raw messages - one entry per UID supplied
			// This gets the raw email data from the pop server for whichever
			// emails are in the UIDList
	s.msgSrcArray = s.popAccount.GetRawMessageByUID(s.UIDList);

			// Turn the array into a query of email structures
	s.messagesQry = s.popAccount.parseRawMessage(s.msgSrcArray);

			// Loop over the array of email messages
	for( s.i=1; s.i <= s.messagesQry.recordcount; s.i=s.i+1) {

			// Process the message as you wish
			// ....
		
		writeOutput("<br/>" & s.messagesQry.subject[s.i]);


			// Now, delete this email from the server 
			// Note: you can also delete all the emails in your query 
			// in one go after the loop. See below.

			// The UID is NOT present in the messagesQry. Therefore,
			// you need to look up the UID by list position.
			// Obviously, you can also selectively delete single messages 
			// based on your criteria.

		s.uid = listGetAt(s.UIDList, s.i, s.mySeparator);
		s.popAccount.deleteMessagesByUID(s.uid);

	}

			// Uncomment to delete all messages in the list in one go, 
			// instead of one by one as above
	//s.popAccount.deleteMessagesByUID(s.UIDList);
}

</cfscript>

POP CFC has lots of other useful functions too.

Cheers,
Murray

Advertisements

ColdDuck – beautiful documentation for ColdFusion CFCs

Background

I have been a consumer of the Sencha ExtJS documentation for a while now and wanted to be able to use JSDuck for my ColdFusion CFCs. But how? While I am a JSDuck novice I figured that trying to make JSDuck produce documentation directly from ColdFusion CFCs was likely to result in tears (mine!). So, after some thought, I came up with another strategy that users of other programming languages might want to consider.

My CF to JSDuck strategy

Mark Mandel’s ColdDoc can be extended to produce different kinds of output by utilising a “strategy” cfc. It comes with one that produces the JavaDoc style HTML format. Strategies utilise the CF ComponentMetaData. So, I made my own strategy that takes ColdFusion CFCs and makes a pseudo-app in JavaScript code that is annotated using JSDuck formatting. Of course it auto-documents functions etc and picks up the ‘hint’ attributes where it finds them and uses all that to produce the raw material for rich documentation. It also works with CF ORM CFCs (although this part could be made even richer than it is at present).

Video overview

Here is a really quick video to give an overview of what ColdDuck does: http://www.screencast.com/t/5p4JqbvNR

Documentation

The ColdDuck package includes the documentation. You can also view it here http://murrah.com.au/coldduck/docs

Sample output

Here is the sample SuperBlog app documented with ColdDoc in JavaDoc format

Here is the sample SuperBlog app documented with ColdDuck in JSDuck format

A full implementation of JSDuck style documentation is here http://docs.sencha.com/extjs/4.1.3/

Almost all of that JSDuck functionality is available for you to use for your ColdFusion projects. I say “almost” because there are a few JSDuck features that are JavaScript specific (eg view JS class source).

Installation

Get it at GitHub https://github.com/murrah/ColdDuck

I hope you find this useful and fun. Please leave your comments below.

Thanks,
Murray

Solved: The CFML compiler encountered an unexpected java.lang.StringIndexOutOfBoundsException exception.

Just a quick one. I received the following error while working a a Coldfusion entity cfc.

The CFML compiler encountered an unexpected java.lang.StringIndexOutOfBoundsException exception.

The line it pointed to as the error condition had no errors in it.

It turned out the the problem was that just before the error line I had an empty comment (an artifact of testing) like /**/ ie no space between the comments “braces”. After changing it to /* */ the error is not thrown. Just in case someone else encounters this message.

500 Null errors with POIutility – java.lang.OutOfMemoryError

I use Ben Nadel’s great POIUtility custom tags to create Excel spreadsheets in Coldfusion MX 7. I have two applications that run on a shared server. Occassionally, the spreadsheet generation stops with the dreaded 500 Null error and I can finally shed some light on the problem. I can often rerun the same generation process with the same data and sometimes it works, sometimes not. Frustratingly, I could not get it to fail on my local development machine until today – when I threw a much larger dataset at it and finally it crashed with the 500 Null error.

So, digging into my exception.log fileĀ  I found what I had suspected was probably happenning on the shared server but couldnt proveĀ  – java.lang.OutOfMemoryError. My service provider said they couldnt see any errors in the log files which wasnt helping my debug efforts!

The final spreadsheet is 1,491 kb on the drive and is 187 columns x 380 rows = 71,060 cells.

I set up some tests on my local development machine and using the same dataset, adjusted the JVM settings and restarted CF each time to work out which setting was the problem. Here are the results:

JVM Heap size Mb MaxPermSize Mb Worked?
512 64 No
750 64 No
750 512 No
925 64 Yes
925 512 Yes
1024 64 Yes
1024 128 Yes
1024 512 Yes

So, it looks like the heap size is the critical setting.

Using CreateObject("java","java.lang.Runtime").getRuntime() and maxMemory = runtime.maxMemory() it looks like the server setting is probably 512Mb.

Interestingly, I also used the java runtime.freeMemory() in the cell.cfm custom tag (using cflog) whenever a new cell object was created to monitor the memory useage and even when the 500 Null errors occurred the memory always seemed to be at least approx 28 Mb and when the last log entry was written each time there was substantially more than that! I guess I was expecting the freeMemory to fall to something close to zero. My guess is that the garbage collection or some other process must have been grabbing some memory?

Now, I just need to convince my service provider to increase their settings. Or, find another service provider!

I hope this helps others with a similar problem.

Cheers,
Murray

CF UDF countArbitraryDaysExcluding

My countArbitraryDaysExcluding UDF has been released on CFLIB.org. You can find it here:

http://www.cflib.org/udf/countArbitraryDaysExcluding

See the comments there. But basically, count all the days in a date range (eg all Mondays) but exclude certain dates (eg for public holidays). These exclusions may be specific dates or patterns ( eg YYYY-1-1 ) – New Years Day in any year so you dont need to update your code once a pattern is defined.

Problem with CF7 dateDiff function – use countArbitraryDays instead

The CF7 livedocs dont mention anything about this so I added a comment. This is more on the subject.

There doesnt seem to be a way to DateDiff with weekdays (ie dont count weekends).

The docs say:
w: Weekdays (same as ww)
ww: Weeks

ie w and ww are synonymous, both mean weeks. That is true when you test it (see below).

The docs for DateAdd however say:
w: Weekday
ww: Week

ie w is days and ww is weeks! Inconsistant.

This code generates the output shown below:

<cfset date1 = CreateDateTime(2009,10,15,21,00,00)>

<cfoutput>date1=#date1#</cfoutput>

<table border=1>
<tr>
	<td>i</td>
	<td>Day() of<br>dateAdd('d',i,date1)</td>
	<td>Day() of<br>dateAdd('w',i,date1)</td>
	<td>dateDiff('d',date1,date2)</td>
	<td>dateDiff('w',date1,date2)</td>
</tr>

<cfloop index="i" from="0" to="10">
	<cfoutput>
		<cfset date2=dateAdd('d',i,date1)>
		<tr>
			<td>#i#</td>
			<td>#day(dateAdd('d',i,date1))#</td>
			<td>#day(dateAdd('w',i,date1))#</td>
			<td>#dateDiff('d',date1,date2)#</td>
			<td>#dateDiff('w',date1,date2)#</td>
		</tr>
	</cfoutput>
</cfloop>

</table>

date1={ts ‘2009-10-15 21:00:00’}

i Day() of
dateAdd(‘d’,i,date1)
Day() of
dateAdd(‘w’,i,date1)
dateDiff(‘d’,date1,date2) dateDiff(‘w’,date1,date2)
0 15 15 0 0
1 16 16 1 0
2 17 19 2 0
3 18 20 3 0
4 19 21 4 0
5 20 22 5 0
6 21 23 6 0
7 22 26 7 1
8 23 27 8 1
9 24 28 9 1
10 25 29 10 1

As you can see, the dateDiff(“w”…) does what the docs say but not what you might expect.

And dont forget that for DateDiff and DateAdd a “day” is a 24 hour period, so dateDiff(‘d’,{a date at 11pm},{the next day at 6am}) returns 0, not 1.

So, my solution (after testing various options including my own) was to use this http://www.cflib.org/udf/countArbitraryDays which seems to work correctly. It is also correct (IMJ) where the first date is (say) 11pm and the second date is at (say) 6am ie not whole 24 hour periods. countArbitraryDays counts these as whole days even if they are not 24 hours apart. That is what I wanted – if it is the next day, it is 1 day irresepective of whether a full 24 hours have elapsed.

Now, back to the job!

#Value! error when using builtin POI in ColdFusion MX7

CF 7 (and 8, I think) has the java POI library builtin which means that (amongst other things) you can create native Excel spreadsheet files. cfSearching and Ben Nadel’s great blogs have more details.

I used Ben’s great POI Utility which is a set of custom tags that allows easy creation of XLS files with CSS styles for formatting. Fantastic stuff. However, when I created a cell containing the Excel COUNTIF() function, the cell displayed the #Value! error even though there was no error. By double clicking the cell then pressing enter, the error went away and the correct result was displayed.

After some googling I discovered that there was a bug in earlier versions of the POI java library that is shipped with CF7 (and 8??). So, nothing to do with the POI Utility, everything to do with the underlying Java.

Time to get the latest version of the POI Java jars and use Mark Mandel’s brilliant JavaLoader.cfc to load them into CF. If you need to know how to do that, see cfSearching’s post on that.

Once installed, the question was then how to change Ben’s POI Utility custom tags to use the new version of the jars instead of the builtin version. What I did was the following which solved the problem. I should say at the outset that I am not fluent in cf custom tags (I use objects instead) so there might be a more elegant way to acheive this.

First, add the following function to the POI/document.cfm page:

<cffunction name="getJavaClass" returntype="any" hint="" access="public">
	<cfargument name="javaLoader" type="any" required="yes" hint="">
	<cfargument name="className" type="string" required="yes" hint="">
 <cfscript>
	if (isObject(arguments.javaLoader)) {
		return arguments.javaLoader.create(arguments.className);
	} else {
		return CreateObject("java", arguments.className);
	}
</cfscript>
</cffunction>

This function will be used to load the java jars using the javaLoader if it is passed in, or the normal CreateObject if there is no javaLoader passed in.

Then, in poi/document.cfm:
after

		<cfparam
			name="ATTRIBUTES.Style"
			type="string"
			default=""
			/>

add the following attribute and variables declaration:

		<cfparam
			name="ATTRIBUTES.javaLoader"
			type="any"
			default=""
			/>

		 <cfset variables.javaLoader = ATTRIBUTES.javaLoader>

Then, change all occurrances of

CreateObject( "java",
eg
<cfset VARIABLES.WorkBook = CreateObject( "java", "org.apache.poi.hssf.usermodel.HSSFWorkbook" ).Init(

to

getJavaClass(ATTRIBUTES.javaLoader,
eg
<cfset VARIABLES.WorkBook = getJavaClass(ATTRIBUTES.javaLoader, "org.apache.poi.hssf.usermodel.HSSFWorkbook" ).Init(

Now, in poi/cell.cfm, change:

CreateObject( "java", "org.apache.poi.hssf.util.Region" ).Init(

to

VARIABLES.DocumentTag.getJavaClass(VARIABLES.DocumentTag.javaLoader, "org.apache.poi.hssf.util.Region" ).Init(

That fragment uses the function and variables that are located on the document.cfm page.

Finally, when you use the poi:document tag on your page, pass the instance of your javaLoader, eg:

<poi:document
	javaLoader="#variables.javaLoader#"
	name="REQUEST.ExcelData"
	file="#xlsfname#"
	style="font-family: verdana ; font-size: 10pt ; color: black ; white-space: nowrap ;">

After those small changes, the #Value! error went away! Yay!!

Thanks again to Mark, Ben and Leigh for making all this possible in the first place.