SurfControl Issues - Updated Apr. 5, 2007

Apr. 5, 2007 - SurfControl 6.0 Memory Usage / Notes on New Version (6.1) Update - As of April 5, 2007, my SurfControl database (sys:etc\cpfilter\SurfControl Categories.cdb) files was at version 1714, and the file size had grown to 790,331kb in size. In order to get this to load in my BorderManager 3.8 (NW 6.5SP6, BM38SP5) server, I had to preallocate 900mb of RAM in the CSPCONFIG.INI file AND I had to load NetWare with a -u665000000 parameter in autoexec.bat. I also had to reduce my NSS cache balance percent to free up RAM, which I did by setting cache balance percent to 15. My server, which only has to support 1-2 users in my office at any one time, has just 1.5GB of RAM. This amount of RAM is barely enough to support SurfControl v6.0. Monitor reports 392,082 original cache buffers, but only 28,000 total cache buffers once SurfControl loads.

So - in order to use the currently-shipping SurfControl, you need:
a. a ton of RAM for it,
b. a -u memory allocation parameter in autoexec.bat, (this value is suggested in NoRM)
c. a memory preallocation value in the 900-1000MB range in CSPCONFIG.INI

Now let's take a look at a new version of SurfControl (v6.1) that I am just starting to beta test.

I downloaded a file from SurfControl (you can probably get it too by emailing them, but I think it may be released by the end of April). The file is a Windows executable, and running it just goes through the usual SurfControl installation process. It took my old sys:\etc\cpfilter directory, and renamed it to sys:\etc\SCV6_1.bak. It then created a new sys:\etc\cpfilter directory, copied my old registration and cspconfig.ini files into it, and created a new sys:\etc\SurfControl Categories.cbd file. The new file was only 377,198kb in size, and was at version 1705. (A word to the wise here - be sure you have enough room on the SYS volume to add another cpfilter directory and do the updates). I have heard that the database was completely rewritten to take less memory, and I can believe it, since the new version is at about the same category list revision as my 6.0 SurfControl at half the database size. I ran CSP_LIST to update SurfControl to the latest category list, and the update worked, bringing it up to revision 1715 in about an hour and a half.

Next, I tested to see if I could load the database with a lot less RAM. I removed the -u665000000 parameter completely from my autoexec.bat file. I remarked out the preallocation parameter in the cspconfig.ini file, and I rebooted the server.

The 6.1 version of SurfControl came up just fine - allocated 524MB of RAM to itself, and did not need the -u parameter or a cspconfig preallocation. This may change eventually, but it looks very good for now.

What about new SurfControl features? I was mistakenly thinking additional categories were to be added with version 6.1, but SurfControl tells me that is not the case. The only change is in memory management and reduction of the size of the database file.


Nov. 11, 2006 - SurfControl Updates no longing working - As was noted in the March 30, 2006 update, SurfControl has been taking HUGE amounts of RAM to cache the database as it grows with each update. What started out (in NBM 3.7) as a 250MB database that would never grow up to 512MB has turned into a RAM-gobbling monster, requiring (currently) upwards of 650MB!. If you have not manually updated the CSPCONFIG.INI file with enough RAM preallocation, at some point your updates will fail because you do not have enough RAM to hold the updated database in memory.

You can see the last time your SurfControl database was updated by looking at the date of the big .CDB file in sys:\etc\cpfilter\cpfilter. You can see when the database failed to update by looking at the end of the cpfilter.log file in the same directory.

I am currently allocating 700mb of RAM for caching SurfControl, and hoping this will be enough for a while. This effectively means you can no longer run a NW 6.5, BM 3.8 server on only 1GB of RAM - you will want to have at least 1.25GB, and preferably 2GB (or more). You can see the syntax for allocating the RAM below.

[SurfControlGeneralList]
CDB_PREALLOC_SIZE=700


March 30, 2006 - SurfControl Updates no longing working - For some time now, the default size for SurfControl updates has not been enough to allow the updates to complete. Lately, I see increasing numbers of people who have been setting the limit WAY up there, in the hopes that we will finally not have to keep creeping the number up. (What happens is that the database gets so large that one update pushes it past the limit of allocated RAM, and the updates stop at that point. This is clearly seen in the cpfilter.log file, if you look). So, if you have plenty of RAM, you might want to just allocate 512MB of RAM for the SurfControl database in the SYS:ETC\CPFILTER\CSPCONFIG.INI file as shown below.

[SurfControlGeneralList]
CDB_PREALLOC_SIZE=512


July 2, 2005 - SurfControl Updates no longing working - In the past couple of weeks, the downloaded updates for SurfControl have grown the SurfControl database so large that it no longer fits into the default memory allocation, and so new updates are not applied. You need to allocate additional memory in the SYS:ETC\CPFILTER\CSPCONFIG.INI file as shown below. I bumped mine from 350MB (default) to 400MB. (Go up to at least 360).

[SurfControlGeneralList]
CDB_PREALLOC_SIZE=400


July 29, 2004 - Just made some corrections to verbiage below. The latest version of SurfControl is Ver. 6, not SP3, and it added 10 categories from version 5 (SP2) not 8.

May 15, 2004: SurfControl has released Version 6. Version 6 has 10 additional categories (40 total), now matching the Windows version of SurfControl. The other settings are essentially the same as Service Pack 2. You should be able to download the latest version, install it one the BorderManager server over the top of the version 2, and just have new categories show up in the access rules menu. If updating previous versions of SurfControl, you may want to delete the old access rule and replace it, and be careful that you are using the latest NWADMN32 snapins and BorderManager patches, especially if you run NWADMN32 from another server.

Sept. 24, 2003: SurfControl has released Service Pack 2.

You need to go to www.surfcontrol.com/downloads, and log in to their system. You can then download Service Pack 2 for BorderManager. The download is 63MB in size.

Service Pack 2 Updates:

The Following notes apply to SurfControl Service Pack 1 - Some Aspects May or May Not Apply to Service Pack 2

Applies to BorderManager 3.7, using the SurfControl version of CPFILTER.NLM, not the older CyberPatrol version.

Memory Allocation Issue with NetWare 6.0

The shipping version of CPFILTER.NLM (installed from CP_SETUP.EXE) has a problem when used in NetWare 6.0. (This does not seem to be a problem in NetWare 5.1). If you run the CSP_LIST.NLM to update the SurfControl categories, huge chunks of RAM disappear for an extended period. I have seen servers with 1GB of RAM run out of memory, getting 'allocator out of memory' messages. The update process can take quite a while, but if / when it completes, the 'lost' memory is eventually recovered. SurfControl has a patched version of CPFILTER.NLM dated April 12, 2002 (ver. 5.00d) which they call Service Pack 1. In order to run the update process you need to get at least an evaluation serial number from SurfControl's web site. If you enter a correct email address, you should immediately receive an email from SurfControl telling you about a patch to download, and instructions for installing the patch. SurfControl does not tell you WHY there is a patch, but this is the reason.

CSP_LIST Update Issue 1

In some circumstances, you can see a problem where the SurfControl database update process fails, with the following entry in the CPFILTER.LOG file:

[Thu May 30 18:05:16 2002] Starting SurfControl Content Database
[Thu May 30 18:05:30 2002] Beginning LiveUpdate procedure
[Thu May 30 18:05:30 2002] LiveUpdate: Current Database Version is 378
[Thu May 30 18:05:37 2002] LiveUpdate: Incorrect syntax near the keyword 'ORDER'.
Statement(s) could not be prepared.

The problem is rather odd, and is related to the version of CPFILTER.NLM running, and a DNS PTR record for your BorderManager server public IP address. If the public IP address has a PTR record pointing to a name that is more than 36 bytes long, the above error can occur, and the update fails. (This can also happen if you have a long name for the server's public IP address in the HOSTS file on the server).

SurfControl is aware of the issue as of early June 2002, and a patch will be coming out. I have tested a beta patch which fixes the problem. I do not know at this time how SurfControl will version, advertise or supply the patch.

CSP_LIST Update Issue 2 (updated Apr. 26, 2003)

For several months, CSP_LIST updates worked OK, gradually updating the SurfControl database file and making it a little larger on each update. However, the database finally became so large that the default memory allocation setting for CPFILTER was too small. CSP_LIST updates would then fail. Thankfully, the CPFILTER.LOG file gives you the answer. You must add an entry in the SYS:ETC\CPFILTER\CSPCONFIG.INI file. I guessed at a value to use, and the following has worked for me (as of Apr 24, 2003)

[SurfControlGeneralList]
CDB_PREALLOC_SIZE=270

Note: For the first part of January 2003, I had set that size to 195 instead of 205. That worked up to a point, and then 196 was required. By Early April, I had to increase the value to 260, then 270.

While at BrainShare, I talked to SurfControl at length on the issues listed on this page. Apparently, some extra, and unnecessary data got into the database update in the March / April timeframe, and it swelled the database file size considerably. This size increase (my .CDB file went from about 150MB to over 183MB) caused a RAM increase requirement. By the time I got back from BrainShare 2003, the problem had been corrected, and the database was pruned. On April 24, my database was at version 656, and was down to 151MB in size. I have left my RAM preallocation at 270, to cover further increases in the future, but if you are tight on RAM, you can reduce the allocation value somewhat. As of April 26, 2003, my .CDB file is now down to 147MB in size, at version 667.

Access Rule Issue (Solved, Aug. 7, 2002)

There is some circumstance whereby the SurfControl categories refuse to display in the access rule for it. Even if the categories were there initially, they may disappear, making control of the rule impossible. In my server, as of June 12, 2002, I have been unable to get the categories to show up, even when I reinstalled SurfControl from scratch.

I noticed this at first when I suddenly could not browse to web sites that I used to. When I checked the SurfControl access rule, there were no categories showing up at all.

The SYS:ETC\BORDER\ENGLISH\SCONTROL.ACL file was suspiciously truncated. (This file is rewritten when CPFILTER loads). Under the [Content Database] section, there were no entries at all.

The problem appears to have been corruption of the SYS:ETC\PROXY\SURFINFO.DAT file. When I deleted that file, along with the SYS:ETC\BORDER\ENGLISH\SCONTROL.ACL file, SurfControl categories suddenly appeared again in my access rule.

If the above doesn't help you, try reinstalling the snapins from the SYS:\PUBLIC\BRDRMGR\SNAPINS\SETUP.EXE file. Or reinstall SurfControl, as noted below.

April 26, 2003: Note that the BM37FP3.EXE file has a RESTRICT.DLL snapin that you have to manually copy to the snapins directory, and would be overwritten with an older version if you ran the snapin setup program

SurfControl Won't Load, Error Message on Server Console (Apr. 1, 2003)

If you cannot get SurfControl to load properly, and you see the following error messages on the server:

ACLCHECK-3.70-16
Content Filtering Solution from Surf Control can not be launched.
ACLCHECK-3.70-21
Error initializing Third Party URL Filtering Solution

You may have experienced a failed update attempt leaving problem files behind in the SYS:ETC\CPFILTER directory. Delete any .TMP files found there and try to reload CPFILTER, or reboot the server.

SurfControl Loads, But Doesn't Work - Uses Very Little RAM (Apr. 1, 2003)

If you experience problems loading SurfControl CPFILTER where it loads but uses only a few K of RAM (normally usess 183MB-205MB of RAM), you may need to change the load order in AUTOEXEC.NCF. CPFILTER should be loaded before BorderManager proxy. That is, load CPFILTER before STARTBRD / BRDSRV.

If you still can't get it to load properly, try loading CSP_LIST instead, as it will try to autoload CPFILTER. This will kick off an update process, but generally works.

The CPFILTER process may not load the actual database into RAM unless it is 'registered' with an Access Rule, or perhaps with CSP_LIST.

SurfControl Access Rule Selects All (or Most) Categories (Apr. 8, 2003)

I don't have an answer for this issue yet, but I have seen it at a client with BM37SP1 installed. You select a SurfControl category, save the changes, and then look and find that all, or almost all categories are selected. You cannot get the rule to select only the desired categories. In my case, a backrev of RESTRICT.DLL did not help, nor did reinstalling SurfControl. When the answer to this issue come out, I will post it here.

Apr. 25, 2004 Update:

There seem to be two schools of thought on this one. One is a complete reinstall, while the other (which makes the most sense to me) is to be sure the snapins are correct. Here are two things to try:

1. Be sure you have the correct snapins. At least you should have (after BM37SP1 or later patches) the choices for None, SurfControl, N2H2 and Connectotel at the bottom of the access rules menu. If you don't see that, you definitely have the wrong snapins. In any case, try running the snapin copy program SYS:PUBLIC\BRDRMGR\SNAPINS\SETUP.EXE. April 26, 2003: Note that the BM37FP3.EXE file has a RESTRICT.DLL snapin that you have to manually copy to the snapins directory, and would be overwritten with an older version if you ran the snapin setup program.

2. Complete reinstall of SurfControl (and do #1 as well). See the following issue for some detail on a complete reinstall.

SurfControl Access Rule Causes ABEND (Apr. 26, 2003)

This one happened on my main Internet server immediately after applying NW51SP6.EXE. The server was at BM37SP2 (with BMMACSSL1 proxy version) and was at NW51SP5 before I applied the new patch on Apr. 23, 2003. I think in retrospect that the SP6 patch probably was not a root cause, but instead some file may have gotten corrupted.

Symptom: BorderManager loaded, then a <1> ABEND occurred shortly afterward. The abend.log file showed

Server BORDER1 halted Wednesday, April 23, 2003 7:52:25.829 pm
Abend 1 on P00: Server-5.00k: Page Fault Processor Exception (Error code 00000000)

and later in that log...

Running process: ACLCHECK 1 Process
Created by: NetWare Application
Thread Owned by NLM: ACLCHECK.NLM

It didn't take long to find out the the ABEND only happened when I had a SurfControl access rule in place. The ABEND would occur shortly after ACLCHECK read the rules at BorderManager startup, or after I added back in a SurfControl rule. Prior to this, the server had been quite stable, and SurfControl had been working well.

I tried a variety of fixes, but ultimately did a complete reinstall of SurfControl to solve the issue. (Note: Or so I thought! The abend came back several hours after I posted this the first time. I may now have an answer - Apr. 25, 2003 - It appears my .CDB database was corrupted. I downloaded a fresh one from SurfControl, and my server has been up 6 hours without an abend since then. Apr. 26, 2003 - It's been about 24 hours since I put in the fresh .CDB file, and still no abend. Last night, the .CDB file was updated by CSP_LIST from version 654 to 667. During that process, it dropped in size from 180MB to 147MB.)

Here are the steps I took to Reinstall SurfControl:

  1. Backed up my SurfControl Categories.CDB file. The file was 151MB in size, and was the 'current' version (656), according to my CPFILTER.LOG file. If I did not back up this file, I would have been starting from version 378 and going through a long update process later. (As it turned out, the file seems to have been corrupt anyway. Moral: have multiple backups of different versions of the file.)
  2. Backed up the CPFILTER.NLM file, which was the Service Pack 1 version from SurfControl. (Actually, I renamed my entire CPFILTER directory, to be sure I was starting fresh, and keeping old files and registration data).
  3. Deleted the SYS:\ETC\BORDER\ENGLISH\SCONTROL.ACL file. This file is overwritten ever time CPFILTER loads, but I wanted to be sure things were starting up clean.
  4. Deleted the SYS:\ETC\PROXY\SURFINFO.DAT file. I suspect something may have been wrong with this file.
  5. Unloaded CPFILTER, deleted any ∓ all old SurfControl rules, and set NWADMN32 to use Connectotel as the 3rd party rule choice in the Access Rules menu. (I have Connectotel's LinkWall running on my server).
  6. Closed NWADMN32.
  7. Reinstalled SurfControl from the SYS:\SURFCONTROL\CP_SETUP.EXE file.
  8. Copied back the Service Pack one version of CPFILTER.NLM to the SYS:\ETC\CPFILTER directory. (My file date is 5/9/2002, but MODULES reports it as Version 5.00d, April 12, 2002).
  9. Loaded SYS:ETC\CPFILTER\CPFILTER.
  10. Reinstalled the BorderManager snapins using the SYS:\PUBLIC\BRDRMGR\SNAPINS\SETUP.EXE program, and launched NWADMN32 at the end of that process. (April 26, 2003: Note that the BM37FP3.EXE file has a RESTRICT.DLL snapin that you have to manually copy to the snapins directory, and would be overwritten with an older version if you ran the snapin setup program.)
  11. Added a SurfControl Access Rule in NWADMN32. The default (non-registered) choices showed up. I picked a choice as a test and saved the rule. ACLCHECK read the rule, and I did not have an ABEND.
  12. Reregistered SurfControl using the SYS:ETC\CPFILTER\REGISTER.EXE program.
  13. Went back to my Access Rule, and all the categories were present again. I selected my usual categories, and saved the rule. Once again, ACLCHECK read the rules, and no ABEND occurred. (Looking good!)
  14. Copied back my version 656 .CDB file, overwriting the fresh install (378) version. Unloaded ∓ reloaded CPFILTER. Refreshed Server in Access Rule menu in NWADMN32, and ... no ABEND. (Woohoo!)
  15. Ran CSP_LIST to try an update. Took two attempts, but came back and reported that the database version was already the latest version as of 4/24/2003.
  16. Tested a few blocked sites, and they were successfully blocked.
  17. April 24, around midnight: The abends came back, something was still wrong here. Removed the SurfControl access rule again.
  18. April 25, 9:00am: SurfControl gives me a URL to download the latest .CDB file. Downloaded it, unzipped it into place, and rebooted the server. 6 hours later, no abends, so I am hopeful that the server is stable again.
  19. April 26, 10:00am: No ABENDs with SurfControl rule in place and CPFILTER running fine. CSP_LIST updated the .CDB file from version 654 to 667 overnight, and everything is looking fine.

I don't really know what happened here, but I suspect the procedure above will work for a variety of situations. It may be that the SURFINFO.DAT file was corrupted, or a snapin issue occurred, or the .CDB database file was corrupted, or all of the above. The reinstall and .CDB file replacement has me going again though.

Note: If you do not have a good backup of the Surfcontrol Database.cdb file, you can call SurfControl technical support, and they should be able to give you a temporary URL that you can use to download the latest revision of the database. (This will be a large download, on the order of 64MB or so...) SurfControl prefers not to publish a URL for anyone to use to download the file, so you will have to call to get it. The alternative is to run the CSP_LIST process and wait for the updating to complete. Be advised now that it is a fine idea to periodically back up that file so you don't have to start at the original version after a reinstall. Just copy the newer version to the CPFILTER directory, and reload CPFILTER.NLM to use the new data. CSP_LIST will pick up its update from the version in that file and go on from there.

Short Term Memory Allocator Is Out Of Memory (May 13, 2003)

If you get errors on the system console about being out of memory, and you actually have enough RAM in the server, it may be related to a problem with a SurfControl update. Check the CPFILTER.LOG file in the SYS:ETC\CPFILTER directory for error messages. If you have any 0 byte .TMP files in the CPFILTER directory, you will definitely need to delete those. I have not found the cause of this problem yet, but it occurs now and again on my server, even though I have plenty of free RAM for the CSP_LIST update to finish. Usually, I find and delete several 0 byte files, and reboot the server, and it is fine again for days or weeks. I have a feeling that the memory issue may come up only after CSP_LIST fails to update and leaves several 0 byte .TMP files in the CPFILTER directory, causing some other issue.

SurfControl Not Blocking All Restricted Sites (May 31, 2003)

After installing BM 37FP3, some sites may not be blocked by SurfControl (Same issue with LinkWall). I found that www.sex.com was not blocked by the adult / sexually explicit category in my SurfControl rule. I'm trying to track this issue down.

There have been reports that you can fix the issue by back-revving ACLCHECK.NLM to the version from BM37SP2, but I just tested that and it did not work here. I tried back-revving ACLCHECK to BM37SP1, and that DID work, but I assume brings back whatever bugs were present in that version of ACLCHECK. (Keep your eyes open for BM37FP4 or some other new patch).

A forum user reported that back-revving ACLCHECK.NLM, PROXY.NLM and RESTRICT.DLL to the BM37SP2 versions fixed his problems.


Return to the Main Page