Up to Something

A New Client

A client turned up our first full-time Mail.app Mac user with Snow Leopard today. I was called in because of attachment sending problems. It seems that files around 7MB would attach and send, but anything larger was failing. The entrenched support reported watching logs, etc. IIS was returning a 401 then a 500 for the sessions that failed, and there was no clear reason.

Troubleshooting

After a little inspection, I thought it might be the request size / http post size. After a quick verification, i determined that the registry limit was not interfering. I next examined the web.config, located at C:\Program Files\Microsoft\Exchange Server\ClientAccess\exchweb\EWS on the server in question. Sure enough, at the bottom:

    </customErrors>
    < httpRuntime maxRequestLength="819200" />
  </system.web>
</configuration>

I increased the limit, and restarted IIS for good measure. The attachments which the server balked at now sent easily and quickly. Yay!.

Errata

During these fateful events, while working and trying to send a 150MB iso to another mail box, Mail.app went nuts and began eating RAM. In less then one minute, it had consumed 1.2GB and swap began on my poor Mac. Quick fingers and a killall Mail in a ubiquitous terminal window solved the problem.

For certainty, I wiped my Mail directory from my Library (I don't use Mail.app for this reason). Problem Solved.

Question

Why is this setting so obscure in Exchange?

Lesser Apps

Applications are like work habits or practices or anything else in life. The ones you have seem like the best when you find them. I have discovered - over and over - that my own desire for excellence is easily pleased but never satisfied. The darker side of this ambition leads to perfectionism, and the absence of it leads worthlessness. In both extremes you'll find identical results: mediocrity, insignificance and long-term failure.

Evolution

I have been using MenuMeters for quite some time and I enjoyed it. Although the software answered questions and needs since it's install, it has never advanced. In fact, according to the FAQ, nothing new has happened since January 2006. I have been displeased with the lack of new features, or improved utility, or any sort of ongoing growth in the software. However, the fact that it worked and I didn't have to answer the problem put it into the "works-even-if-it's-not-ideal" category.

Change

Snow Leopard brought out incompatibility, and with that it was time for change. I uninstalled MenuMeters prior to installing SNeopard, and therefore had no side effects. However, my O.C.D. need to bury my menu bar in seas of icons and text would not rest. Deliverance in the means on an email from a friend arrived this morning.

Answer

The interesting folks over at iSlayer have a variety of useful tools, though this is my first experience with them. I downloaded and installed iStat Menus. It's a beautiful thing. I don't know if it leaks memory, I don't know if it farks with SpotLight, I don't even know if it's going to crash when I sleep my Mac. What I do know is this.

• It has all the graphs, measures and charts which I required.
• It finally works with temperature. (Something which MenuMeters hadn't since Leopard.)
• It shows definite signs of a tool which is moving and not stagnate.
• It is free.
• It's better in every way - so far.

I can always put up a post about the horrors of pogrom and ghettos if it turns out to be an evil socialist application. (e.g. if Safari needs 500M so do I). However, the first blush indicates that i should have looked over there sooner!

Muchas Gracias Señor Piñera for your recommendations.

I have to say something

The most annoying feature to geek-users of Safari is the practice of uncompressing downloaded .gz files. I find the whole auto-unzip generally useful. It's a problem whenever I do things like downloading the latest pfSense. I need the original compressed tarball. It has a checksum which must match for clean upgrades. As we all know, Safari helpfully uncompresses it an leaves the tarball - a waste - for me.

Today I forgot the special right-click-Download-file step and ended up with more .tar files. There was a new behavior though. The original .gz files were in the Trash! That was good thinking. Not sure if this is a Safari change or a Sneopard change, but regardless. I like it.

Hurray for the last bastion of un-themeable GUIs and retarded iPhone Application Rejection policies. They got one thing correct.

Additional Note

I recovered roughly 14GB of disk space on my MacBookPro when I installed Sneopard. impressive. 14 gb. I just keep wondering: What the hell did it remove? I have not seen or heard of anyone else recovering so much. The best other than me was 12GB, so far. That is doing it right as well.

I have a client with a 17" MacBookPro "MBP", and he one day begin experiencing a login/bootup problem. Another client in the same network hit one of those annoying OpenDirectory login expirations which locked him entirely out of his laptop. (APPLE FAIL). It was the same day, the same network, and similar symptoms (to the end user at least). So, I passed the 17" MBP one off to my counterpart, and I dealt with reseting the AuthenticationAuthority of the latter.

The 17" MBP suddenly booted - ONCE. After that, it was never meant to be. Upon careful investigation we discovered the following alarming facts

• The drive was slowly disintegrating and produced all sorts of weird problems.
• The User(tm) had not been getting good backups for Some Weeks(c).
• The backups, did not include the users home directory. (WAT?)
• The Users home directory. It's encrypted.

Enough suspense. We now have a Perfect Storm of exciting problems. This user is THE key person in all things related to product design and development in a company. Only the Good Lord knows how much important data there is in the 41GB of encrypted volume.

The Testing

My first problem was the problem with "Mac". HFS+ is not grok'd by anything except commercial disk utilities and Mac OS X. Mac OS X, being clever, auto-mounts everything on startup. We had to have a dirty shutdown of the filesystem, and the farking journal is in an area of the disk which has gone bad. I need to mount the filesystem, read-only, no recovery, no journal.

I know that it's at least still got basic constructs. The boot process starts working and Single User Mode will almost start. Modules are clearly getting loaded, etc, etc, etc.

Disabling Services

I grab my 15" MBP, and set to work on figuring out how to disable the automount behavior. I tried a few mac peeps whom I know. No one knows anything. I tried service centers. They know even less than my people. At length, I discovered and disabled the following services, with this handy script:

#!/bin/bash

launchctl unload /System/Library/LaunchDaemons/com.apple.metadata.mds.plist 
launchctl unload /System/Library/LaunchDaemons/com.apple.autofsd.plist 
launchctl unload /System/Library/LaunchDaemons/com.apple.automountd.plist 
launchctl unload /System/Library/LaunchDaemons/com.apple.diskarbitrationd.plist 

• ...metadata.mds.plist -- Spotlight Indexing engine.
• ...autofsd.plist -- Who cares, it's 'auto' and 'fs' in the same line.
• ...automountd.plist -- Who cares, it's 'auto' and 'mount' in the same line.
• ...diskarbitrationd.plist -- This is the king pin. Stop it, and Disk Utility fails, and everything auto-* stops.

Cataloging Disk Devices

I booted up, plugged in my recovery disk, saw that it was available in Finder and at /Volumes/LonghornData and I knew I was ready to proceed.

I wanted a simple, verifiable way to check which disks where what on my system. So, I checked /dev for any 'disk' entries, and saved them into a TextEdit session. For some strange reason, my laptop, and my main recovery-volume presents this way:

Tulkas:log root# ls -l /dev/disk?
brw-r-----  1 root  operator   14,   0 May 11 21:32 /dev/disk0
brw-r-----  1 root  operator   14,   3 May 11 21:50 /dev/disk3

After connecting the 17"MBP, I found a new device - /dev/disk4 - and it has two slices, as expected:

brw-r-----  1 root  operator   14,   5 May 12 09:03 /dev/disk4
brw-r-----  1 root  operator   14,   7 May 12 09:03 /dev/disk4s1
brw-r-----  1 root  operator   14,  14 May 12 09:03 /dev/disk4s2

Armed with enough information to proceed, I moved on.

Practical recovery

I placed the 17"MBP in Target Disk Mode and connected up to my laptop. The files were located on the volume in /Users/.username. So, I mounted the disk as such:

Tulkas:Volumes root# ls -l /dev/disk?
brw-r-----  1 root  operator   14,   0 May 11 21:32 /dev/disk0
brw-r-----  1 root  operator   14,   3 May 11 21:50 /dev/disk3
brw-r-----  1 root  operator   14,   5 May 12 09:32 /dev/disk4
Tulkas:Volumes root# mkdir /Volumes/D1/
Tulkas:Volumes root# mount_hfs -o rdonly -j /dev/disk4s2 /Volumes/D1/
Tulkas:Volumes root# cd /Volumes/D1/Users/.myuser/myuser.sparsebundle/bands

After that, it was a step-by-step use of the above procedures, with "umount -f D1" after I/O Stalls which saved me. The drive made it through recovery of all 41GB of data files. As far as I can tell, it looked to be 100% success. At the time of writing, I haven't verified the encrypted data, but, that is of lesser importance (for a blog article) than such gems as DiskArbitrator and stopping auto-mount/recovery of filesystems.

Read any post you like. Read any place you like. In 1994 it was "NTFS doesn't fragment by design, it never fragments". In the later 90's it was "ext3" or some other filesystem "doesn't fragment". Later, starting in 2000 - OS X doesn't fragment.

Windows

It started for me Windows in about 1994, and hit a sort of minor crescendo around 2002. I was speaking - ON THE PHONE - with Microsoft Technical Support. They were telling me about how fragmentation doesn't exist on NTFS, and that the new object linking would allow persistent data across an entire network. Whatever that means. NTFS not only fragments, but it HYPER fragments. There are some excellent write ups on this phenomena, but it underlines a few points. The first is that designing fragmentation proof systems is a process of revision, at least in part, and not necessarily all "design".

Unix Derivatives

Pearls of Wisdom dispensed at the tail ends of sanity constantly admonish n00bs that Mac or Linux won't fragment, even if of other platforms will. Old school boards and such used to offer identical advice to the WindowsNT crowd. The sad facts are in. EVERYBODY FRAGMENTS. It's a function of longevity, slowly growing files, LARGE files in general and use patterns with the disk. Filesystems fragment very rapidly when you start running out of space, and no matter how large the volume, space eventually gets exhausted.

Leopard

Leopard does well, but even it has limits. My 700mb Thuderbird inbox has well over 4000 file fragments. (I run out of space a lot). I have several .DS_Store files which have more than 3 fragments. If you have space, if it can be done fast and if it's under 20mb, Leopard will defrag most any file on the fly. That said, if you get low on space, make your computer extra busy, and then handle large files, you're doomed.

The unspoken dilemma

One of the things which Windows Defragmenters have taught us is the value of reorganizing the disk layout. I can't tell you if it's honestly that valuable, but I can tell you if gives a warm fuzzy feeling.

There may be many ways and reasons which filesystems fragment, but they ALL fragment. People who tell you otherwise are clueless or stupid. The presence of software to defragment is nice and necessary for filesystem health. Get it. Love IT. USE IT. Profit.

_{This article is in response to all the clueless n00bs, drinking Apple's Koolaid and spouting ignorant stupidity as though it were lucid rational.}

MacOSX Strives to be more rigthteous than the next bloke:

Dec  1 20:38:22 xXxMac SyncServer[2205]: SyncServer: Truth vacuumed. 
Next vacuum date 2008-12-15 20:38:21 -0800

Wah-Hoo

• Flickr Example
• Apple regular discussion
• Yet Another Apple Discussion
• Apple Development Explains the Truth Database. (WTF is the truth?)

OpenVZ can have various heinous problems with udev. Most often, you cannot enter the VZ from the admin, and you cannot connnect via SSH.

beast / # vzenter 51
enter into VE 51 failed
Unable to open pty: No such file or directory

In Fedora you can make a simple change to /etc/udev/makedev.d/50-udev.nodes

--- 50-udev.nodes~      2008-01-10 16:00:08.000000000 +0000
+++ 50-udev.nodes       2008-07-30 15:44:07.223092644 +0000
@@ -1,4 +1,5 @@
 # These device have to be created manually
+ptmx
 tty1
 tty2
 tty3

e.g. Simply just add 'ptmx' to the file someplace. This is fairly simple and seems to work very well.

Most people think that bash.org is just a collection of useless quotes. IRC is a seething underbelly of stupidity and intellectual mayhem. However, it appears that that the vacant stares actually assist in the discovery of blinding insights.

Clintons

McCain (only one)

Republicans in general

Democrats Candidates in general

America in general

There are more Bush related quotes than can be read in a single day. (Finding them is the difficult part, bash.org people have lots to say about 'bush' as opposed to Bush.) Cheney quotes are fairly prolific as well. However, I think that this quote really sums up the feelings of the community and the moderators for the current administration.

In short, according to Bash.org: Democrats are racists or sexists, Republicans aren't smart, (as Bush would put it:) "the current administrations is idiots", oh, and we're all doomed.

Why is it that normal people don't run more successful projects? See this ticket regarding Paludis. Paludis is a C++ replacement for Portage. Portage is a squirrels nest and even though the ebuild system is pretty nice, portage itself is pretty lame.

Disclosure

In the interest of fairness, I would point out: The Paludis site flatly refuses to say anything about the project. Therefore being run by a complete asshole isn't contradictory to any previous statements.

Summary

The gist of the ticket is that Paludis doesn't support certain types of parallelism and the developer(s) refuse to do anything sane in order to prevent, notify or clearly document the danger of it. This danger is apparently readily realized by users.

Examples of sanity might be:

• Warn people it's not supported.
• Make some sort of method for restricting parallel execution.
• ADD A NOTE TO A FAQ

Evidence

What does the Paludis think of a notice about the dangers of parallel runs of the software they publish:

chaoflow: "What about preventing parallel paludis runs or at least a FAQ or some other way of explicitly telling people, that parallel paludis operations are not supported?"
ciaranm: "Preventing parallel runs is a security hole. And an FAQ entry is pointless -- parallel executions are fine so long as they stick to certain operations."
chaoflow: "Wouldn't it be nice to have documented, which operations are fine for parallel execution?"
ciaranm: "Not really. If you don't already know, you shouldn't be doing it at all."

Very Dense

This is clearly beyond the scope of Paludis. See this conversation:

chaoflow: "What about preventing parallel paludis runs?"
ciaranm: "Preventing parallel runs is a security hole."
chaoflow: "Could you elaborate on that?"
ciaranm: "It's an inversion. A non-root user can obtain the locks and prevent root from being able to do anything for an arbitrarily long time."

This stunning display of logic and intellect is what passes for success over at Paludis. Even I can think of a few methods to help prevent this:

• Make an override mechanism. Easy enough, right?
• Enable super-users to kill the offending process.
• Put the lockfile in a secured-location.
• Use shared memory, and make use of ipcrm to kill rogue locks.
• Observe that a security /hole/ involves some sort of exploitation of a system. A DoS involves prevention of normal operation. This doesn't even make a legitimate DoS.

Blackhole

The stupidity doesn't end there:

chaoflow: "And way way better would be some simple locking inside of paludis preventing bad things from happening."
ciaranm: Paludis is not there to protect you from yourself.

ciaranm must see this as an incredibly clever way of saying "go f*** yourself.". ciaranm seems to be an incredibly dense idiot. Why is an f'd up system preferable to some logic which may lead to the inconvenience of cleaning up a rogue lock?

Paludis IS a security hole - It just might fuck up your system, if you run it in parallel with itself, but it certainly won't try to tell you that. I am guilty too.

In order to get Ahsay working behind and SSL proxy which passes traffic to port 80, you have to modify your conf/server.xml and set a few settings on ol' Nginx.

Add to your server.xml, non-SSL connector declaration

scheme="https" secure="false" proxyPort="443" redirectPort="443"

nginx config section

    proxy_pass      http://127.0.0.1:9080;
    proxy_redirect  http://archive.myisteam.com      https://archive.myisteam.com;
    proxy_redirect  http://archive.myisteam.com:80   https://archive.myisteam.com;
    proxy_redirect  https://archive.myisteam.com:80  https://archive.myisteam.com;

.....

Apart from that, it's perfectly normal

Special thanks to Cliff Wells. For Tireless effort in the face of java.

Thanks as well to the Apache Documentation efforts. Tomcat Connector Docs