CFPOP – deleting an email when there is a comma in the UID

It seems that a problem has existed with the CFPOP delete action for a number of versions and it has not been fixed. The problem occurs when trying to delete a single email (or a list of emails) from the mail server where the mail server has included a comma in the UID. eg “1382396111.12731.mydomain.net,S=965”

The CF docs for the CFPOP Delete action says: “UID or a comma-separated list of UIDs to get or delete. Invalid UIDs are ignored.”

So, the abovementioned UID gets split into 2 uids (notice that it has a comma in it) and of course the email is not located so cant be deleted. CFPOP does not include a DELIMITERS parameter to use with the UID paramater which would solve the problem, I think. eg

 
<cfset myUIDlist = "1382051456.15221.mydomain.net,S=965|1382051456.18000.mydomain.net,S=966" />  
<cfpop server = "********" username = "********" password = "******** 
	action = "Delete" uid="#myUIDlist#" delimiter="|" >

Fortunately, there is a solution – use POP CFC instead of CFPOP! Thanks to Paul Vernon.

So, here is a sample page to demonstrate a way to use POP CFC.

<cfscript>
/**
 * Example of reading, processing then deleting individual emails from the pop mail server
 * using the POP CFC custom component.
 * http://popcfc.riaforge.org/
 */

	// To make it easier to rescope this example (local, variables ... whatever)
variables.s = {};

			// Initialise the custom POP CFC component. Used instead of CFPOP
			// Set your values here
s.popAccount = createObject("component", "pop").init(myPopServerHostname, myUsername, myPassword)

			// Specify a non-comma separator for the UID lists. 
			// You must do this in order for the list functionality
			// of POP CFC to work (including deletions)
s.mySeparator = "|";
s.popAccount.setProperty("separator",s.mySeparator);

			// Get an array of the UIDs of the messages in the mail account.
			// ie retrieve them from the mail server
			// This is a necessary step to solve the comma-in-uid problem
s.UIDArray = s.popAccount.getUIDList();

if (arrayLen(s.UIDArray)) {
			// Convert the array to a list
			// Keep the list to use for deletions
	s.UIDList = ArrayToList(s.UIDArray, s.mySeparator);

			// Get an array of the raw messages - one entry per UID supplied
			// This gets the raw email data from the pop server for whichever
			// emails are in the UIDList
	s.msgSrcArray = s.popAccount.GetRawMessageByUID(s.UIDList);

			// Turn the array into a query of email structures
	s.messagesQry = s.popAccount.parseRawMessage(s.msgSrcArray);

			// Loop over the array of email messages
	for( s.i=1; s.i <= s.messagesQry.recordcount; s.i=s.i+1) {

			// Process the message as you wish
			// ....
		
		writeOutput("<br/>" & s.messagesQry.subject[s.i]);


			// Now, delete this email from the server 
			// Note: you can also delete all the emails in your query 
			// in one go after the loop. See below.

			// The UID is NOT present in the messagesQry. Therefore,
			// you need to look up the UID by list position.
			// Obviously, you can also selectively delete single messages 
			// based on your criteria.

		s.uid = listGetAt(s.UIDList, s.i, s.mySeparator);
		s.popAccount.deleteMessagesByUID(s.uid);

	}

			// Uncomment to delete all messages in the list in one go, 
			// instead of one by one as above
	//s.popAccount.deleteMessagesByUID(s.UIDList);
}

</cfscript>

POP CFC has lots of other useful functions too.

Cheers,
Murray

ColdDuck – beautiful documentation for ColdFusion CFCs

Background

I have been a consumer of the Sencha ExtJS documentation for a while now and wanted to be able to use JSDuck for my ColdFusion CFCs. But how? While I am a JSDuck novice I figured that trying to make JSDuck produce documentation directly from ColdFusion CFCs was likely to result in tears (mine!). So, after some thought, I came up with another strategy that users of other programming languages might want to consider.

My CF to JSDuck strategy

Mark Mandel’s ColdDoc can be extended to produce different kinds of output by utilising a “strategy” cfc. It comes with one that produces the JavaDoc style HTML format. Strategies utilise the CF ComponentMetaData. So, I made my own strategy that takes ColdFusion CFCs and makes a pseudo-app in JavaScript code that is annotated using JSDuck formatting. Of course it auto-documents functions etc and picks up the ‘hint’ attributes where it finds them and uses all that to produce the raw material for rich documentation. It also works with CF ORM CFCs (although this part could be made even richer than it is at present).

Video overview

Here is a really quick video to give an overview of what ColdDuck does: http://www.screencast.com/t/5p4JqbvNR

Documentation

The ColdDuck package includes the documentation. You can also view it here http://murrah.com.au/coldduck/docs

Sample output

Here is the sample SuperBlog app documented with ColdDoc in JavaDoc format

Here is the sample SuperBlog app documented with ColdDuck in JSDuck format

A full implementation of JSDuck style documentation is here http://docs.sencha.com/extjs/4.1.3/

Almost all of that JSDuck functionality is available for you to use for your ColdFusion projects. I say “almost” because there are a few JSDuck features that are JavaScript specific (eg view JS class source).

Installation

Get it at GitHub https://github.com/murrah/ColdDuck

I hope you find this useful and fun. Please leave your comments below.

Thanks,
Murray

Solved: The CFML compiler encountered an unexpected java.lang.StringIndexOutOfBoundsException exception.

Just a quick one. I received the following error while working a a Coldfusion entity cfc.

The CFML compiler encountered an unexpected java.lang.StringIndexOutOfBoundsException exception.

The line it pointed to as the error condition had no errors in it.

It turned out the the problem was that just before the error line I had an empty comment (an artifact of testing) like /**/ ie no space between the comments “braces”. After changing it to /* */ the error is not thrown. Just in case someone else encounters this message.

500 Null errors with POIutility – java.lang.OutOfMemoryError

I use Ben Nadel’s great POIUtility custom tags to create Excel spreadsheets in Coldfusion MX 7. I have two applications that run on a shared server. Occassionally, the spreadsheet generation stops with the dreaded 500 Null error and I can finally shed some light on the problem. I can often rerun the same generation process with the same data and sometimes it works, sometimes not. Frustratingly, I could not get it to fail on my local development machine until today – when I threw a much larger dataset at it and finally it crashed with the 500 Null error.

So, digging into my exception.log fileĀ  I found what I had suspected was probably happenning on the shared server but couldnt proveĀ  – java.lang.OutOfMemoryError. My service provider said they couldnt see any errors in the log files which wasnt helping my debug efforts!

The final spreadsheet is 1,491 kb on the drive and is 187 columns x 380 rows = 71,060 cells.

I set up some tests on my local development machine and using the same dataset, adjusted the JVM settings and restarted CF each time to work out which setting was the problem. Here are the results:

JVM Heap size Mb MaxPermSize Mb Worked?
512 64 No
750 64 No
750 512 No
925 64 Yes
925 512 Yes
1024 64 Yes
1024 128 Yes
1024 512 Yes

So, it looks like the heap size is the critical setting.

Using CreateObject("java","java.lang.Runtime").getRuntime() and maxMemory = runtime.maxMemory() it looks like the server setting is probably 512Mb.

Interestingly, I also used the java runtime.freeMemory() in the cell.cfm custom tag (using cflog) whenever a new cell object was created to monitor the memory useage and even when the 500 Null errors occurred the memory always seemed to be at least approx 28 Mb and when the last log entry was written each time there was substantially more than that! I guess I was expecting the freeMemory to fall to something close to zero. My guess is that the garbage collection or some other process must have been grabbing some memory?

Now, I just need to convince my service provider to increase their settings. Or, find another service provider!

I hope this helps others with a similar problem.

Cheers,
Murray

CF UDF countArbitraryDaysExcluding

My countArbitraryDaysExcluding UDF has been released on CFLIB.org. You can find it here:

http://www.cflib.org/udf/countArbitraryDaysExcluding

See the comments there. But basically, count all the days in a date range (eg all Mondays) but exclude certain dates (eg for public holidays). These exclusions may be specific dates or patterns ( eg YYYY-1-1 ) – New Years Day in any year so you dont need to update your code once a pattern is defined.

Problem with CF7 dateDiff function – use countArbitraryDays instead

The CF7 livedocs dont mention anything about this so I added a comment. This is more on the subject.

There doesnt seem to be a way to DateDiff with weekdays (ie dont count weekends).

The docs say:
w: Weekdays (same as ww)
ww: Weeks

ie w and ww are synonymous, both mean weeks. That is true when you test it (see below).

The docs for DateAdd however say:
w: Weekday
ww: Week

ie w is days and ww is weeks! Inconsistant.

This code generates the output shown below:

<cfset date1 = CreateDateTime(2009,10,15,21,00,00)>

<cfoutput>date1=#date1#</cfoutput>

<table border=1>
<tr>
	<td>i</td>
	<td>Day() of<br>dateAdd('d',i,date1)</td>
	<td>Day() of<br>dateAdd('w',i,date1)</td>
	<td>dateDiff('d',date1,date2)</td>
	<td>dateDiff('w',date1,date2)</td>
</tr>

<cfloop index="i" from="0" to="10">
	<cfoutput>
		<cfset date2=dateAdd('d',i,date1)>
		<tr>
			<td>#i#</td>
			<td>#day(dateAdd('d',i,date1))#</td>
			<td>#day(dateAdd('w',i,date1))#</td>
			<td>#dateDiff('d',date1,date2)#</td>
			<td>#dateDiff('w',date1,date2)#</td>
		</tr>
	</cfoutput>
</cfloop>

</table>

date1={ts ‘2009-10-15 21:00:00’}

i Day() of
dateAdd(‘d’,i,date1)
Day() of
dateAdd(‘w’,i,date1)
dateDiff(‘d’,date1,date2) dateDiff(‘w’,date1,date2)
0 15 15 0 0
1 16 16 1 0
2 17 19 2 0
3 18 20 3 0
4 19 21 4 0
5 20 22 5 0
6 21 23 6 0
7 22 26 7 1
8 23 27 8 1
9 24 28 9 1
10 25 29 10 1

As you can see, the dateDiff(“w”…) does what the docs say but not what you might expect.

And dont forget that for DateDiff and DateAdd a “day” is a 24 hour period, so dateDiff(‘d’,{a date at 11pm},{the next day at 6am}) returns 0, not 1.

So, my solution (after testing various options including my own) was to use this http://www.cflib.org/udf/countArbitraryDays which seems to work correctly. It is also correct (IMJ) where the first date is (say) 11pm and the second date is at (say) 6am ie not whole 24 hour periods. countArbitraryDays counts these as whole days even if they are not 24 hours apart. That is what I wanted – if it is the next day, it is 1 day irresepective of whether a full 24 hours have elapsed.

Now, back to the job!

#Value! error when using builtin POI in ColdFusion MX7

CF 7 (and 8, I think) has the java POI library builtin which means that (amongst other things) you can create native Excel spreadsheet files. cfSearching and Ben Nadel’s great blogs have more details.

I used Ben’s great POI Utility which is a set of custom tags that allows easy creation of XLS files with CSS styles for formatting. Fantastic stuff. However, when I created a cell containing the Excel COUNTIF() function, the cell displayed the #Value! error even though there was no error. By double clicking the cell then pressing enter, the error went away and the correct result was displayed.

After some googling I discovered that there was a bug in earlier versions of the POI java library that is shipped with CF7 (and 8??). So, nothing to do with the POI Utility, everything to do with the underlying Java.

Time to get the latest version of the POI Java jars and use Mark Mandel’s brilliant JavaLoader.cfc to load them into CF. If you need to know how to do that, see cfSearching’s post on that.

Once installed, the question was then how to change Ben’s POI Utility custom tags to use the new version of the jars instead of the builtin version. What I did was the following which solved the problem. I should say at the outset that I am not fluent in cf custom tags (I use objects instead) so there might be a more elegant way to acheive this.

First, add the following function to the POI/document.cfm page:

<cffunction name="getJavaClass" returntype="any" hint="" access="public">
	<cfargument name="javaLoader" type="any" required="yes" hint="">
	<cfargument name="className" type="string" required="yes" hint="">
 <cfscript>
	if (isObject(arguments.javaLoader)) {
		return arguments.javaLoader.create(arguments.className);
	} else {
		return CreateObject("java", arguments.className);
	}
</cfscript>
</cffunction>

This function will be used to load the java jars using the javaLoader if it is passed in, or the normal CreateObject if there is no javaLoader passed in.

Then, in poi/document.cfm:
after

		<cfparam
			name="ATTRIBUTES.Style"
			type="string"
			default=""
			/>

add the following attribute and variables declaration:

		<cfparam
			name="ATTRIBUTES.javaLoader"
			type="any"
			default=""
			/>

		 <cfset variables.javaLoader = ATTRIBUTES.javaLoader>

Then, change all occurrances of

CreateObject( "java",
eg
<cfset VARIABLES.WorkBook = CreateObject( "java", "org.apache.poi.hssf.usermodel.HSSFWorkbook" ).Init(

to

getJavaClass(ATTRIBUTES.javaLoader,
eg
<cfset VARIABLES.WorkBook = getJavaClass(ATTRIBUTES.javaLoader, "org.apache.poi.hssf.usermodel.HSSFWorkbook" ).Init(

Now, in poi/cell.cfm, change:

CreateObject( "java", "org.apache.poi.hssf.util.Region" ).Init(

to

VARIABLES.DocumentTag.getJavaClass(VARIABLES.DocumentTag.javaLoader, "org.apache.poi.hssf.util.Region" ).Init(

That fragment uses the function and variables that are located on the document.cfm page.

Finally, when you use the poi:document tag on your page, pass the instance of your javaLoader, eg:

<poi:document
	javaLoader="#variables.javaLoader#"
	name="REQUEST.ExcelData"
	file="#xlsfname#"
	style="font-family: verdana ; font-size: 10pt ; color: black ; white-space: nowrap ;">

After those small changes, the #Value! error went away! Yay!!

Thanks again to Mark, Ben and Leigh for making all this possible in the first place.

Looping over a query with CFScript

It is often convenient to be able to loop over a coldfusion query with CFScript. There is a way to do this even though CF7 does not provide a documented way to do it (and I dont know about CF8).

By harnessing the underlying Java you can do the following:

while( qry.Next() ) {
    qryRow = queryRowToStruct(qry);
    writeOutput( "name=" & qryRow.firstname );
}

(subsititute your relevant column name for “firstname”)

The Next() method is a Java iterator method and can be used on CF queries.

The queryRowToStruct() function is a modified version of the one found at
CFLib.org

The purpose of the function is to return a CF structure containing the data at the specified query row number. See the link above for details.

You just need to modify the line
var row = 1;
to
var row = query.getRow();

getRow() is another Java iterator method and it returns the sequential row number from the query. The Next() method increments the row counter on the query and since the query is passed into the queryRowToStruct() function by reference, each call the getRow() will return the current row number, starting at 1.

Addendum: In order to allow the queryRowToStruct() function to be used as it was originally intended, you need to make another small change so that you can use the function without the iterator and only one parameter ie so that queryRowToStruct(query) will return the first record from the query.

Here is the complete (modified) function (credits to Nathan Dintenfass (nathan@changemedia.com)):


function queryRowToStruct(query){
	// By default, do this to the first row of the query
	//var row = 1;
	// No. By using getRow() we can call this function sequentially and get the next record each time.
	// Or, we can pass the row number we want as the 2nd parameter as before.
	var row = query.getRow();
	//a var for looping
	var ii = 1;
	//the cols to loop over
	var cols = listToArray(query.columnList);
	//the struct to return
	var stReturn = structnew();

	// getRow() will return 0 if next() hasnt been called. 
	// So, to allow this function to work by calling it with one parameter and no previous call to next(), 
	// we need the following:
	if (row eq 0) row = 1; 

	//if there is a second argument, use that for the row number
	if(arrayLen(arguments) GT 1)
		row = arguments[2];
	//loop over the cols and build the struct from the query row
	for(ii = 1; ii lte arraylen(cols); ii = ii + 1){
		stReturn[cols[ii]] = query[cols[ii]][row];
	}		
	//return the struct
	return stReturn;
}

You can also still do this
qryRow = queryRowToStruct(query,3);
to get the third row (as per the original use of the function).

For other info on using Java and queries see:
href=”http://www.bennadel.com/blog/204-Using-ColdFusion-Query-s-Underlying-Java-Methods-For-Query-Manipulation-And-Logic.htm”

Ext – Adding a tree to the portal example

For a project I am working on I wanted to add a tree to a viewport. There is a portal example on the Ext web site that features an accordion control in the west panel. It took a while to figure out how to replace that accordion with a tree even though the method turned out to be quite simple.

See this post on the Ext forum for the key piece of information from another Ext user:
http://extjs.com/forum/showthread.php?t=14445

See this working version (and files to download) on my web site:
http://www.murrah.com.au/samples/ext/portal/portal.cfm

Ext JS library and Coldfusion

I am enjoying using the Ext JS 2.0 library and at the same time finding it very frustrating due to a lack of well documented examples. The library is a very comprehensive object oriented Javascript library that is aimed at building data rich web applications.

To help others (especially Coldfusion programmers like me) I have started an Ext & Coldfusion wiki page on the Ext site that will, I hope, become a central place for CF programmers to leave tips about how to integrate CF and Ext.

The wiki is at http://extjs.com/learn/Manual:Resources:ColdFusion