USB 3.0 works under Linux

Sunday, August 22nd, 2010

I decided that I needed a real backup solution, even though I have a RAID 5 for file storage in my workstation; maintaining a backup of a 2TB array is a pain in the ass if all you have is blank DVDs.

So, I purchased a Vantec NexStar 3 SuperSpeed (NST-380S3) enclosure, a Samsung EcoGreen F3EG 2TB 5400rpm (HD203WI) drive, and a USB 3 PCI-E controller.

It seems the only shipping USB host controllers the moment all use NEC’s USB 3.0 chip, and almost all the PCI-E boards look alike. They all seem to run in the $25-45 range. The great part is Linux supports NEC’s controller as of 2.6.31. The controller worked with no configuration soon as I put the card in.

I chose that specific Samsung drive because it seems to be the only sane 5400 rpm 2TB drive out there. The only other choices were Seagate’s new 5900 rpm drives (which, according to independent reviews on Newegg and enthusiast forums have an unacceptably high failure rate, very unusual of Seagate), and Western Digital’s Caviar Greens (which are 5400 rpm, but suffer from obsessive head parking which is apparently leading to premature drive failure).

Several reviews peg the HD203WI at an average of 90mb/sec writes for sequential writing, or about 2-3x the speed of USB 2.0.

mkfs.ext4 took 7:44 minutes to create the file system (while iotop confirmed it was doing in excess of 100mb/sec writes for much of the process), and hdparm -t /dev/sdx also indicates the drive in this enclosure can push 100mb/sec.

After writing to the drive for an hour straight, the enclosure is warm but not hot, and after removing the drive from the enclosure, the drive itself is warm; this is compared to the Seagate 7200.12s in my RAID 5 array which could burn you at this point.

Many drives fail in enclosures because they overheat; I don’t think this will happen due to Vantec’s thick aluminum design in the NexStar series enclosures, and the fact that the HW203WI has low power usage.

After formatting with ext4, the file system uses 29GB out of 1.82TB total. Its kind of funny when I’ve owned drives smaller than the space consumed by an empty file system.

I’m rather happy with my purchases overall.

FGLRX Catalyst 10.6 has slight bug

Friday, June 18th, 2010

I’m not sure how many people this bug effects, but with certain setups Catalyst 10.6 will segfault on load. This effects very few people.

The fix is to edit /etc/X11/xorg.conf and add BusID "PCI:1:0:0" inside the Section "Device" that defines your card (almost all people only have one of these)

The BusID should match the PCI ID that lspci | grep Radeon says.

Approximate Youtube Bitrates

Monday, May 24th, 2010

I’ve been wondering what bitrates Youtube produces on files, but they don’t upfront say.

New videos are encoded in eight formats. However, due to bug in Youtube, some 24 fps videos (such as those from film sources) will have duplicate frames inserted to make them 30 fps, causing a very noticeable jitter approximately twice a second.

Format Video Codec Audio Codec Container
37 H.264 1920×1080 24/30 fps AAC 44.1khz Stereo mp4
22 H.264 1280×720 24/30 fps AAC 44.1khz Stereo mp4
35 H.264 854×480 24/30 fps AAC 44.1khz Stereo flv
34 H.264 640×480 24/30 fps AAC 44.1khz Stereo flv
18 H.264, 480×360 24/30 fps AAC 44.1khz Stereo mp4
5 Sorenson Spark, 320×240 24/30 fps MP3 22khz Stereo flv
17 MPEG-4 ASP, 12 fps, black bordered to fit 176×144 frame AAC 22khz Mono mp4
13 H.263+, 15 fps, stretched to full frame 176×144 ignoring source aspect ratio AMR 8khz Mono 3gp

Note: This does not include WebM videos yet as the support is still experimental, and Youtube is not yet encoding videos in 1080p, only 720p (format 45) and 480p (format 43).

Now lets see how a couple high quality videos fair on Youtube.

Format Resolution Video and audio bitrate in kbit/sec
The Dark Knight Trailer 3 1080p, using the Apple version. 2:30 long. H.264, 6ch 48khz AAC audio, 24 fps. Youtube encoded this as a 30 fps video.
Original 1920×816 10518 260
37 1920×816 3427 108.8
22 1280×544 1998 108.8
35 Missing on Youtube

34 640×272 517 95
18 480×204 500 108.5
5 320×136 257 64
17 176×144 55.3 27
13 176×144 55.6 13
Avatar Trailer 1080p, using the Apple version. 3:29 long. H.264, stereo 44.1khz AAC audio, 24 fps.
Original 1920×800 9726 99
37 1920×800 3502 126
22 1280×534 2003 126
35 854×356 806 103.84
34 640×266 554 103.81
18 480×200 486 103.82
5 400×166 255 59
17 176×144 55 28
13 176×144 54 13
Big Buck Bunny 1080p, using the Blender Foundation‘s original version. 9:57 long. Theora, stereo 48khz Vorbis audio, 24 fps.
Original 1920×1080 11902 175
37 1920×1080 3531 125
22 1280×720 2020 125
35 854×480 990 107.9
34 640×360 494 108.02
18 480×270 435 108.03
5 400×226 250 59
17 176×144 55 30
13 176×144 49 12

With these 3 popular HD videos, its easy to tell what sort of bitrate Youtube tries to hit.

Format Approximate bitrate target (video and audio)
37 3.75mbit/sec
22 2.25mbit/sec
35 1.25mbit/sec
34 768kbit/sec
18 768kbit/sec
5 384kbit/sec
17 100kbit/sec
13 75kbit/sec

SLF4J and making JUL shut up

Monday, February 15th, 2010

I’ve decided to switch to the Simple Logging Facade for Java (SLF4J) plus Logback to bridge java.util.logging (JUL), Log4J, and Apache Commons Logging all into one log output.

Problem is, JUL won’t shut up. Frameworks that log to JUL output the log to the console, and then SLF4J repeats it right after. However, putting this code in before running SLF4BridgeHandler.install() seems to fix it:

java.util.logging.Logger root_logger = java.util.logging.LogManager.getLogManager().getLogger("");

java.util.logging.Handler[] root_handlers = root_logger.getHandlers();

rootLogger.removeHandler(root_handlers[0]);

Now I get one single log output alone.

Trying to get OSX to work well in VirtualBox

Friday, February 12th, 2010

Update: This article is out of date. Virtualbox officially supports OSX nearly flawlessly under 3.2: native EFI booting, USB keyboard/mouse, networking, and sound (using a third party driver). The only thing missing is 3D acceleration support.

Now that VirtualBox 3.1.4 is out, I decided to try OSX. I would like to thank this thread on the VBox forums for a few hints on how to do this.

I am doing this on a Core 2 Duo. This should work on any processor that has at least SSE3.

These instructions are for people who are used to using VirtualBox.

You need:

  1. At least VirtualBox 3.1.4
  2. A Hackintosh ISO. I used iDeneb 1.6 Lite, which is based on OSX Leopard 10.5.8. Others may or may not work depending if they include the appropriate hacks.

Make a new virtual machine, set:

  1. Set it for FreeBSD
  2. With at least 1GB of memory
  3. And at least 10GB of HD space

Set the additional settings of:

  1. System->Motherboard->Enable IO APIC to on
  2. Do NOT enable EFI, it is not complete enough for OSX yet
  3. System->Processor->Processor(s) matching what you actually have
  4. System->Processor->Enable PAE/NX to on
  5. Display->Video->128MB of video memory, with 3D Acceleration enabled
  6. Storage->IDE Controller->Type set to ICH6
  7. Network->Adapter 1->Attached to Bridged Adapter, advanced->Intel PRO/1000 MT Desktop (NOT MT Server or T Server)

Exit VirtualBox, edit the XML definition for the machine. For me, and I named my machine OSX, this was under ~/.VirtualBox/Machines/OSX/OSX.xml. Add the line…

<ExtraDataItem name="VBoxInternal2/SupportExtHwProfile" value="on"/>

… inside the <ExtraData> block, and save it and restart VirtualBox.

Mount the iDeneb ISO inside your virtual machine, and start it up. Remember to press enter to start the CD on boot.

Use the disk utility to format your drive to HSF journaled. Select that drive in the installer and hit the customize button before installing.

Select the following options in the iDeneb OSx86 Components menu:

  1. Kernel->Qoopz
  2. Essential Patches->ACPI Fix
  3. Essential Patches->PS/2 Drivers->ApplePS2 Old
  4. Drivers->Chipset->ICHx Fixed

Now, just install. This takes about 20 minutes.

To get network working after boot, go into the client’s menu while running and do Devices->Network Adapters, uncheck Cable connected, hit okay, go back in, and check Cable connected. After that OSX will connect to the network, and you have to do this every time you boot.

Sound cannot be fixed at the current time: VirtualBox’s AC97 emulation supports a version of ICH AC97 too old for the popular third party AppleAC97Audio.kext to support. Hopefully VirtualBox will get a HDA emulation, which will improve chances of being emulated correctly.

2D speed seems to be pretty fast without specific drivers, although some operations will be noticeably slow. I wish there was a specific video driver for this.