Paul Gregg

Jack of all Tech.

Amusing, True Near Death Experience.

Written By: pgregg - Aug• 25•2011

Those who know me will know that there have been two important events occurring in my life this week. 1) Tiling my kitchen, and 2) Trying to buy a HP TouchPad.

As part of tiling the kitchen, significant other wanted to replace all the standard white wall sockets with Stainless Steel ones to match the rest of the kitchen colour scheme. I was safely over half way through this task, having disconnected the socket mains supply, when a van pulled up bearing HP TouchPad goodness. Thus the next 30 minutes of my life was accounted for.

Upon returning to the kitchen tasks I came to the Cooker 40A switch. I disconnected the ring main from the switch and proceeded to move the wires so that I kept the supply and load pairs apart when BUZZZZZZ and 240 Volts of high energy goodness shot through my hand and up my arm.

Yes folks, TouchPad caused me to forget to flip the cooker ring circuit in the fuse box.

Cue me jumping back, dropping tools and collapsing to the floor in a bout of Arrrrgh.

Wife said “Stop it, you’ll scare the kids.”.

Kids run in asking what happened and wife explains that Daddy electrocuted himself.

Six year old Lauren asked “Could you see his bones?”.

What 16TB raw space looks like at home

Written By: pgregg - Apr• 11•2011

I’ve been looking for some home backup solutions over the past couple of months. This has led me down both the do-it-yourself route and buying a ready-made solution.

One of my requirements was that I wanted the solution to be more than just storage – otherwise I would have purchased a straight NAS box from the likes of Qnap, Netgear or if feeling rich Drobo. Most of these dedicated NAS boxes can be “rooted” to allow ssh access , however their CPUs are generally underpowered for general purpose use.

Other requirements were that I wanted a reasonably small form factor and to be able to use at least 4 SATA hard drives, preferably with hot swap ability. Hardware raid was not a requirement because I intended on using a Linux distribution with mdadm software raid.

In the end, I ended up building two boxes.
The first, a home build, based on the CFI A7879 chassis CFI_A7879_1with a Gigabyte GA-D525TUD Dual Core Atom Mini-ITX Board.

GA-D525TUD

The second was a off-the-shelf HP ProLiant Microserver which, to be brutally honest, was because HP were offering £100 cashback deal on it. This made the server much cheaper than you could possibly build yourself from components.

HP_Microserver
I added 4GB ram to each box (total 5GB in the HP box because it comes with 1GB).The CFI boot drive is a 8GB (30MB/sec) CompactFlash card mounted as an
IDE drive. The HP boot drive is a 16GB Sandisk Cruzer USB stick.

Finally added 4 x 2TB Samsung F4EG HD204UI drives to each box.

The CFI box has 8TB in RAID5 providing 5.4TB usable. The HP has 8TB in RAID6 providing 3.6TB usable space.

If there is more interest, I’ll write up the build process is more detail with pictures.

For now – here are some shots of my utility shelf.

IMG_20110411_173250

IMG_20110411_173740

 

 

 

Experts Exchange, Google, AllFAQ.org and misappropriation of copyright.

Written By: pgregg - Jan• 19•2011

Opinion Piece

I was googling (as a verb) and came across a rather peculiar message at the bottom of Google’s search results:

In response to a complaint we received under the US Digital Millennium Copyright Act, we have removed 1 result(s) from this page. If you wish, you may read the DMCA complaint that caused the removal(s) at ChillingEffects.org.


Interesting – never saw that before!

Following the link to Chilling Effects shows a copy of the complaint which has some interesting text in it.

Experts-Exchange makes a detailed itemisation of their registered Copyrights, none of which I find objectionable, however, the complaint then goes on to list several issues against the Defendant, the first and most egregious of which is:

a direct “copy and paste job” lifting the content of Plaintiff’s question and answer forums and inserting them onto AllFAQ’s website. AllFAQ’s question and “Solutions” are verbatim to Experts-Exchange’s questions and “Accepted Solutions;”

From this Experts Exchange is accusing allfaq.org of Copyright infringement against Experts Exchange owned Copyright.

At first glance, this might seem fully justified – but look at what they are claiming copyright on.  Experts Exchange are assuming copyright ownership of content that you, and I, and all their users create by asking and answering questions on their web site.

I looked at Experts Exchange’s Terms of Use and could not find any agreement that users were assigning their rights and copyrights to Experts Exchange. The relevant paragraph is:

“5. Content License

EXPERTS EXCHANGE enables Members to post problems or questions,
proposed solutions or answers, information, comments and other content
(“Your Content”) to its Site. When you post Your Content to the Site,
you understand and agree that Your Content can be viewed and used by
other Members who visit the Site with or without attribution.

You represent and warrant that you own or otherwise control all of the
rights to Your Content and that use of Your Content by EXPERTS
EXCHANGE and its affiliates will not infringe upon or violate the
rights of any third party. Before you use EXPERTS EXCHANGE Services to
post any information or content that is protected by intellectual
property laws, you shall have acquired the legal right to do so from
the owner or authorized licensee of such information or content.

By registering with EXPERTS EXCHANGE and posting Your Content on the
Site, you hereby: (i) grant EXPERTS EXCHANGE a non-exclusive,
perpetual, irrevocable, unrestricted, transferable, fully
sub-licensable, worldwide, royalty-free license to use, distribute,
display, reproduce, perform, modify, adapt, publish, translate and
create derivative works from Your Content in any form, media or
technology, whether now-known or hereafter developed; (ii) grant
EXPERTS EXCHANGE and its affiliates and sub-licensees the right to use
the Member Name that you submit with Your Content for purposes of
attribution; (iii) authorize EXPERTS EXCHANGE to assert and prosecute
claims against any third-party making any unauthorized use of Your
Content, including any use that violates this User Agreement
(“Third-Party Claims”); and (iv) appoint EXPERTS EXCHANGE as your
attorney-in-fact for the purpose of asserting and prosecuting
Third-Party Claims. If you do not wish to have Your Content attributed
to you, then you must notify EXPERTS EXCHANGE at

customer_service@experts-exchange.com
.

Experts Exchange acknowledges that the copyright belongs to the author as “Your Content” and that by posting you are granting them extensive licenses to use that content. You are not assigning your copyright to Experts Exchange.

Now I am glad that their ToU does not attempt to wrest copyright ownership from its rightful owner, that is right and proper.

allfaq.org is demonstrably guilty of screen-scraping the Experts
Exchange web site and I do not condone those actions at all. However, looking at what
they copied – it was the Title, Question and Accepted Solution text –
the copyright of 100% of that is with the original authors, and not
Experts Exchange.

Thus, in my opinion, this complaint against allfaq.org is without merit and should be dismissed.

It would also appear that Experts Exchange has also abused the provisions of the DMCA in forcing Google to remove the content. Google should restore the links.

And finally, Experts Exchange should implement some technical measures to prevent automated scraping. Find better ways to improve your search ranking, and if your competition beats you don’t ask your own members how to do better SEO; be told by them that you have no Copyright Claims on the content; and then proceed to file DMCA take down notices when you know you have no (copy)right.

Stage 2: http:BL with Apache2 mod_perl

Written By: pgregg - Dec• 02•2010

After my earlier post Referrer and Comment spammers are a PITA I came up with two mod_perl plugins to Apache and an “apache level” firewall.

The reason for the apache-level firewall is two-fold.  There is no direct way for the Apache user to manipulate an iptables chain (as it doesn’t run as root), and second; I was not happy with suid root access or other forms of message passing to a daemon which would manipulate the firewall for me.

Architecture is thus, in httpd.conf place the following two lines:

PerlPreConnectionHandler PGREGG::httpBLBlock
PerlLogHandler PGREGG::httpBLLog

The first tells apache to run the handler in my httpBLBlock.pm module when a connection is received (before the request has been sent by the client).  In this handler, I am simply looking for a filename matching that IP in a directory that is writable by the apache user.  The contents of the file are a SCORE:httpBL_answer:[LIST].  Based on this, the module checks the mtime of the filename is in the last SCORE days, then the firewall is in effect. If so, we simply tell apache to drop the connection.  If the file has expired, we delete the file.

The second line is more interesting, and what creates the firewall filenames. In order to not impede the general speed of request handling, processing is performed in the Logging section of the Apache process. Our module is called by apache after the response has been sent, but before the access_log entry has been written.  In our module we perform the http:BL API call and compute the above SCORE based upon the Threat* level and Age* of the API response. (* both Threat and Age are octets in the DNS lookup).  We merely discount the Threat down to zero based on the Age (0-255) where an entry 255 days old reduces the SCORE to zero.
If the SCORE is larger than our trigger level (3) then we create the firewall filename, log the entry in our own httpbl.log and return Apache2::Const::FORBIDDEN.  This causes Apache to not log the entry in the normal access_log.  Otherwise, if all is ok, we return Apache2::Const::OK and Apache logs the hit as normal.

I have a bit of code tidy up, restructure the config/firewall directory and pull some common code out to a shared module before I can release to the world.

An interesting side effect to publishing the last story out through Planet PHP and other news sources along with the Project Honey Pot image is that when browsers viewed those sources, they all asked for the image off my server. In several cases, these were known spammer, Comment spammer, and other abusers. My server then created the firewall entry blocking them before they were able to follow the links back to my server.
 
I have been reading up more on Apache Bucket Brigades in an attempt to allow the firewall filter to be placed immediately after the request has been received and allow a custom response to the browser. This may help an otherwise unsuspecting user if their machine had been trojaned. I don’t mind admitting I’m thoroughly confused right now :)

Referrer and Comment spammers are a PITA.

Written By: pgregg - Nov• 28•2010

This shouldn’t be news to anyone – but Referrer and Comment spammers are a real pain in the a*se.  Polluting my web logs and making any meaningful log analysis problematic.

So, I now have an itch to scratch and I’m going to do something about it. I would encourage you, the reader, to do something about it too.

Firstly, get yourself over to Project Honey Pot and read up on the project.  If you can, set up a Honey Pot or two yourself. Also be sure to read about the http:BL – this works along similar lines to the DNS blacklists used for Email spammers.

Next, I’m going to write a general Apache mod_perl module which will provide integration (lookup) to the http:BL and allow the user to “action”* the abusers.  Minimally, it will prevent the normal apache log files from being polluted by diverting the log entries to a httpbl logfile.

* “action” – To provide flexibility, I’m thinking of running an external script with the IP of the abuser.  The script can then perform any action you wish. The one I’m going for is an iptables firewall block.

Comments and suggestions welcome.

Project Honey Pot has implementations for several languages, including PHP and Perl (the languages that mean most to me).  There may be an implementation for your Web application so you might not be interested in what I’m doing at all :)

14-projhoneypot_banner.jpg

Vodafone UK + HTC Desire + Android 2.2 FroYo = Fail.

Written By: pgregg - Aug• 04•2010

Well, it looks like Vodafone UK royally messed up the timing of the HTC Desire OTA Update yesterday.  While most owners were eagerly expecting the announced Android 2.2 (FroYo) update that HTC have been pushing out, Vodafone decided to push out a 2.1-update1 which only provides Vodafone branding, apps, a few bugs, and even a couple of “adult” related bookmarks to everyone – and leaving them on Android 2.1 (Eclair).

Needless to say – Users are not happy at all. *Vodafone appears to have moved the thread here. [2010/08/06] Moved again to here (is Vodafone trying to hide the complaints?).

In work, we (coworkers and I) now have a total of 8 HTC Desires (out of 16 people) – even two iPhone users have converted! A few have been espousing the wonders of FroYo on their phone (some had rooted, and did it themselves, others had bought unbranded phones and got the stock HTC FroYo upgrade). I was jealous(ish) and wanted it myself.

Given Vodafone’s actions yesterday, it was likely that the FroYo update from Vodafone was minimum several weeks away, if not 3 months (as was intimated on the Vodafone forum – end of October) – it was also noted that the Vodafone FroYo update would include the Vodafone 360 branding and software.

Thus, last night it was time to embark on the adventure of flashing my HTC Desire to a stock image direct from HTC. I reasoned that my goal should be to flash HTC’s Android 2.1 (Eclair) image to the phone, and once there, the normal software update process should take me to 2.2.

And so it began…

I knew that I needed to create a gold card because the Vodafone image doesn’t let you install non-Vodafone images. This I discovered as I tried to simply apply a stock 2.2 download and using boot recovery update.zip – all attempts met with a complaint of a “Fingerprint error”.

I ended up at this page:
  [TUT]Complete upgrading guide(root, unroot, flashing ROM & updates)

and followed Post #3 which details a) How to make a Gold Card* . Ignore the rest of #3
and then Post #4 – the “unrooting” guide.

You then need to find a download of the correct stock HTC Bravo WWE image ROM – I chose the second WWE ROM from this link (140MB download):
  [ROM] Official HTC Desire RUU ROMS and OTA Update URLs
  RUU_Bravo_HTC_WWE_1.21.405.2_Radio_32.36.00.28U_4.06.00.02_2_release_126984_signed.exe

Proceed through the rest of Post #4 “How to Flash ROM” with the Goldcard inside your Desire, and plugged into your PC.

The phone will take 5-10 minutes to complete upgrading (or downgrading) to HTC Stock 2.1 image. When it reboots you will have to go through all the original setup sequences you did when you first unboxed your phone.

You can then perform a Software Update Check and you should find you have a Android 2.2 FroYo update (90Mb) waiting for you.  Proceed and let it do its thing.

Once done, welcome to FroYo.

All future updates will come direct from HTC – not from Vodafone, and you won’t ever have the Vodafone 360 branding rubbish foisted upon you.

* My GoldCard creation had a bit of a hiccup, in that it turns out that the 4GB Samsung card which came with my Desire does not work as a Goldcard despite formatting and following the instructions to the letter.  Trying an old 1GB Sandisk I had resulted in a good goldcard.

Aside from the goldcard hiccup – this all went surprisingly smoothly and painlessly.

Good luck.

Disclaimer: If you try any of the above – it is all your responsibility. I take no responsibility should you brick or damage your phone.

The O2 Joggler – A first hack.

Written By: pgregg - Apr• 14•2010
12-devicev2-thumb-308x149-11.jpg

If you were not aware – O2 last week reduced the price of the O2 Joggler from £149.99 to £99.99 to £49.99.  Nothing remarkable in price reductions, however what is remarkable is what you actually get for your money.

The O2 Joggler is a silent 7″ touchscreen device with Intel Atom Z520 running at 1.3Ghz, 512Mb ram, 1GB internal flash storage with additional storage available via an external USB port. What makes the device really exciting is that it also has a 1GB ethernet (Realtec 8168) and Wifi.

Looking at that – I know you’re all thinking “linux box” – but conveniently, the default operating system on it is based on Ubuntu 8.04 and busybox, The frontend is a custom flash driven UI developed by OpenPeak (makers of the Jogger which is rebadged by O2).

There are plenty of clever individuals over at http://hackthejoggler.freeforums.org/and in the #mer channel on freenode who are working to allow the Joggler to run other Operating Systems such as Android, Ubuntu Netbook Edition, MID, even Windows and OSX.

However, the standard OS supplied is also pretty powerful and allows significant customisation.  You can enable telnet just by turning it on with a custom USB stick plugged in. To do some of the things I’ll be describing here you will need to have started with that.

Once you can telnet in, the world opens up and you can do lots of things that you would expect* to be able to do in a linux system. (* except run a web browser…. at this time – we don’t have web access
with the stock OS due to the custom flash GUI interface.
)

Lets install some useful utilities, how about perl, terminfo, irc clients, bit torrent, rsync, ssh/scp ?   Yes, I know what you’re thinking – overnight silent downloads – no need to leave PC on… nice…

I’ve built several of these utilities with installation instructions and uploaded them over here: 
  http://pgregg.com/projects/joggler/

Feel free to download, examine, install etc.  Read the README on the download page – it explains how to do it. Most of the packs include an install.sh script that will provide an installation safety net – it won’t overwrite existing files or libraries.

Usual disclaimers apply – you try these applications at your own risk. I accept no responsibility if you manage to brick, fry or otherwise trash your joggler.

Comments, suggestions for other applications are always welcome.

Update: Thanks to NP – seems I was a little too brutal with the library stripping to keep the download sizes small and I missed some required libraries (that I mistakenly thought were in the standard Joggler distro).   I have rebuilt the following packages as they were missing some libs: rtorrent, rsync, sudo,ssh
The screen package has been rebuilt too to add one final tweak (to the installer script) so it can be used by non-root users – the only difference you need to do is run the command: chmod 666 /dev/ptmp /dev/tty

Is BT still collecting usage data?

Written By: pgregg - Mar• 31•2010

After the recent scandal over BT using Phorm (see here and here) is it still possible that BT is still covertly collecting data profiling its user’s internet usage?

I have, as part of my home BT Total Broadband package, a HUAWEI Mobile Connect 3G Modem.  This is a USB dongle that provides internet service.   I rarely use it, but because of the storms last night my internet connection has been very unstable, therefore I plugged it in today.

When I started the “BT Connection Manager” software, it informed me that there was an update to the service and would I like to download it? Yes, ok, and a couple of minutes later it began to install itself.  Then popped up the obligatory EULA which contained the paragraphs:

Installation of this Software will automatically apply a unique identifier to the Software, this identifying feature will only be used in accordance with the BT privacy policy which can be found at www.bt.com.

This software automatically collects session and client parameter Information for all connections made via this Software. This information is automatically transmitted to BT at the start of each session where Internet access is available. This session information will only be used in accordance with the BT privacy policy which can be found at www.bt.com.”

BTConnectionManagerEULA.txt

I also note that the software package uses the Open Source packages libCurl, openssl and sqlite.  Combined they enable the package to record “session and client parameter Information for all connections” in the database (sqlite) and to encrypt (openssl) and transmit the data via a web request (libcurl) back to BT.

At this point I am unable to confirm if such a transmission is taking place as I declined the EULA, but would welcome feedback or comments from others that are able to investigate this more fully.

Update: Even though I clicked “Cancel” to reject the EULA – it did update the application and actually started it up before I exited the installer.  Guess I have this privacy invading feature now.

Amazon 1 – 0 Waterstones, aka this morning’s rant.

Written By: pgregg - Sep• 29•2009

SWMBO asked me to order a book for her this morning, so being the tightwad that I am, I go hunting for the ISBN and the cheapest place to buy it.
Amazon has it for £17.24 and Waterstone’s for £18.49 (but if I order through quidco, then I receive another 8% cashback, making Waterstone’s cheaper).

Waterstone’s need me to create an account, fair enough and state “Also, please be aware that passwords are case sensitive, and must be a
minimum of seven characters long and at least one character must be
numeric.”  Again, all fairly standard.

So each time I try to enter a password which meets this criteria I’m hit with a javascript popup that claims: “Your new password must be at least 7 characters long and contain at least one digit.”

“But it is!”, I yell in frustration.

Using the Firefox “Web Developer” plugin I unhide the password boxes and see that my password doesn’t have digits… wtf!   I enter the digits again and find there is another hidden rule:

<input name="newPassword" type="password"
id="fPassword" size="40" maxlength="16" value="" />

Yes, an upper limit of 16 characters on the password.

Right ok, make a password of 16 chars or under and move on…

Add name, postcode to the next form and it finds my address ok – click continue to (I presume) proceed to the CC entry form and voila!

Well not quite.

“There has been a problem processing this request

Please use the refresh button on your browser to try again.

Thank you.”

A few refreshes later, it is apparent that Waterstone’s has no intention of working, sod it, for 20p more I can have less hassle at Amazon.

Getty Images / BBC Math Fail.

Written By: pgregg - Jun• 03•2009

From the BBC Magazine today is an article 7 questions on GCSE maths.

Can you spot the fail?

bbc_math_fail1

Yes, epic.

bbc_math_fail2