Category: BSD

Cleaning up a linux machine (arch)

TLDR

journalctl --vacuum-size=100M #remove all logs, only retain 100mb
pacman -Scc #remove all package installation files (obsolete and current)
pacman -S bleachbit
bleachbit -c system.*

First, what's big on the system

du -d1 -h / 2>/dev/null | sort -h

This shows a sorted list of the largest dirs in `/`
You can do two levels down:

du -d2 -h / 2>/dev/null | sort -h

My result is:

0 /proc
0 /sys
0 /tmp
12K /dev
12K /srv
16K /lost+found
632K /run
4.3M /boot
13M /opt
15M /etc
75M /root
93M /home
2.4G /var
3.2G /usr
221G /mnt
227G /

I ignore /mnt (because that's an external drive)
Two dirs stand to mind: var and usr.
Let's see what's inside:

du -d1 -h /var /usr 2>/dev/null | sort -h

Then, a little bit deeper:

du -d1 -h /var/log /usr/share /usr/lib /var/cache 2>/dev/null | sort -h

Let's start with the logs

I have 717mb in /var/log.

I'm not a fan of deleting directories randomly, so let's do it the clean way:

$ journalctl --disk-usage
Archived and active journals take up 728.7M on disk.

Let's leave only 100mb of logs:

journalctl --vacuum-size=100M
...
Deleted archived journal /var/log/journal/ba5391...b.journal (8.0M).
...
Vacuuming done, freed 616.6M of archived journals on disk.

More info here on how to configure journalctl here.

Packages

I have 660M /var/cache/pacman. It was 1.8gb, but I ran pacman -Sc to remove unused packages. Let's remove the rest:

pacman -Scc

/usr/share/locale

A lot of users do remove it, or at least clean it up. But I might be a problem. But checkout bleachbit (next paragraph).

Bleachbit

Automatic cleaner. Will delete a lot of stuff, but for it was mostly locales.

$ pacman -S bleachbit
$ bleachbit -p system.*
Disk space to be recovered: 488.8MB
$ bleachbit -c system.*

You can look for more stuff to delete:

bleachbit --list
bleachbit -p thunderbird.*

Fosdem 2016

My first fosdem this year. I found a lot of interesting things mostly on the opensource tracks. The "free java" track IMHO was a bit unpopular even though Stephen Chin and Mark Reinhold were there.

I guess I'm used to a better treatment on a conference (I go mostly as a speaker). Here it was rainy and cold and even though the french fries were awesome, I'm used to better food.

IMG_6498 IMG_6499 IMG_6500

Building OpenJDK 9 on a OSX (or any linux)

This is a tiny tutorial on how to build your own copy of OpenJDK 9 from the current sources.

Contents

Preparation
├─Install a package system
├─Install ccache
└─Command Line Tools
Download OpenJDK 9
Build OpenJDK 9
JTREG
├─JTREG: The variables
└─Running tests
webrev

Preparation

Install a package system

Install brew or macports. I prefer brew. Both are package systems for OS X.

Install ccache

ccache is a compiler cache - speeds up the build.

$ brew install ccache

Command Line Tools

We need them so we have a C compiler and other tools.

Install Xcode Command Line tools ( you don't need the whole Xcode, only the command line tools)

Download OpenJDK 9

Get the sources, in the console type:

$ mkdir jkd9
$ cd jdk9
$ hg clone http://hg.openjdk.java.net/jdk9/dev jdk9.hg

Takes 10-20 secs. This downloads basically a couple of scripts that we will execute to get the actual sources.

Now we execute the script we downloaded:

$ cd jdk9.hg
$ chmod u+x get_source.sh
$ ./get_source.sh

The last command can can take 10-20 minutes depending on the connection speed.

Build OpenJDK 9

$ chmod u+x configure
$ ./configure
$ make clean images

The second command took a bit less than 20 minutes. The first command is fast.

The result: we have this created:  build/macosx-x86_64-normal-server-release/images/jdk/ - this is the built OpenJDK 9 - SUCCESS.

JTREG

jtreg is a "testing harness" (whatever that means) made specially for OpenJDK. We need it to run the tests written for the OpenJDK

Download from https://adopt-openjdk.ci.cloudbees.com/view/OpenJDK/job/jtreg/ (not the Snapshot version, the other one).

unzip in jdk9, so we have jdk9/jtreg.

JTREG: The variables

The official instructions suggest we edit ~/.bashrc. I don't like changing global (for your user) settings, so we can put these settings in jdk9/.bashrc

$ nano .bashrc

Add the following lines:

export SOURCE_CODE=/Users/steve/jdk9/
export JTREG_INSTALL=$SOURCE_CODE/jtreg/
export JT_HOME=$JTREG_INSTALL
export JTREG_HOME=$JTREG_INSTALL
export JPRT_JTREG_HOME=${JT_HOME}
export JPRT_JAVA_HOME=${PRODUCT_HOME}
export JTREG_TIMEOUT_FACTOR=5
export PRODUCT_HOME=$SOURCE_CODE/jdk9.hg/build/macosx-x86_64-normal-server-release/images/jdk
export PATH=$PATH:$PRODUCT_HOME/bin:$JT_HOME/linux/bin/

NOTE: Change /Users/steve/jdk9/ with your folder on the top line.
NOTE: Check the correctness of PRODUCT_HOME too.

To execute this file, do:

$ source .bashrc

Running tests

$ cd jdk9.hg/test
$ make jdk_util

Webrev

Webrev crawls over your changes to generate a set of web-based views of the differences in your code. The different views allow reviewers to easily look at your proposed changes, selecting the appropriate difference format based on the type of file, the type of change, and their own preferences.

webrev.ksh is a 70kb kshell script. Pretty weird.

Go to jdk9.hg and:

jdk9.hg# wget http://hg.openjdk.java.net/code-tools/webrev/raw-file/tip/webrev.ksh
jdk9.hg# chmod u+x webrev.ksh

To prepare a webrev, make some changes. For example in the jaxp folder. Then we commit locally in the mercurial repo.

jdk9.hg# cd jaxp
jaxp# hg commit -m "BGJUG: Fix compiler warnings in jaxp repository"
jaxp# ../webrev.ksh

It will prepare a webrev/ dir and webrev.zip. The folder is then uploaded somewhere and sent to the appropriate mail list inside openjdk. Example:
http://bgjug.sty.bz/bgjug/~bgjug/fix-warnings-jaxp-part1/webrev.00/

Note: execution times come are measured on a pretty recent Macbook Air.

Source: https://java.net/projects/adoptopenjdk/pages/AdoptOpenJDKBuildInstructions

[jug.bg] Building Valhalla and using primitives as generic arguments

Project Valhalla will probably be part of Java 10 (expected 2018). Among other things it allows using primitives as generic arguments.

Building Valhalla

On a linux/unix/osx box we start with:

hg clone http://hg.openjdk.java.net/valhalla/valhalla
cd valhalla
bash get_source.sh  #this one takes some time
bash configure
make clean images # this one takes some cpu

Box<int>

We can have a class like this:

public class Program  {
	public static void main(String[] args) {
		// this one still doesn't work, List not updated to List
//		List l = new ArrayList();
//		l.add(5);
//		System.out.println(l.get(0));
 
		Box b = new Box(6);
		System.out.println(b.get());
	}
}
 
class Box {
    private final T t;
    public Box(T t) { this.t = t; }
    public T get() { return t; }
}

javap

Running javap gives

$ javap -c Program.class gives:
[openjdk@localhost bin]$ ./javap -c Program.class 
Compiled from "Program.java"
public class Program {
  public Program();
    Code:
       0: aload_0
       1: invokespecial #1                  // Method java/lang/Object."":()V
       4: return
 
  public static void main(java.lang.String[]);
    Code:
       0: new           #2                  // class "Box${0=I}"
       3: dup
       4: bipush        6
       6: invokespecial #3                  // Method "Box${0=I}"."":(I)V
       9: astore_1
      10: getstatic     #4                  // Field java/lang/System.out:Ljava/io/PrintStream;
      13: aload_1
      14: invokevirtual #5                  // Method "Box${0=I}".get:()I
      17: invokevirtual #6                  // Method java/io/PrintStream.println:(I)V
      20: return
}

Running our pretty simple app gives:

[openjdk@localhost bin]$ ./java Program
Specializing Box${0=I}; searching for Box.class (not found)
Specializing Box${0=I}; searching for Box.class (found)
6
[openjdk@localhost bin]$

Notes: List is still not parameterized with because ... issues.

USB over IP - connect directly to a remote usb device

Problem1: My printer's scanner does not work over the wire with OSX. So the scanner part works by cable only. Well what if I put the usb cable over the wi-fi? Well, now I have a wireless scanner.

Problem2: I have a remote backup drive that I want to keep bootable and encrypted. Was it possible before this? I couldn't make it work. Is it possible now? I think so.

VirtualHere is a commercial software. For free it allows one simultaneous device, if you disconnect the first, you can use the second. It works flawlessly.

Its server has builds for most linuxes plus OpenWrt. The client is very simple, builds for osx, win, linux.

There are FOSS alternatives. If anyone has tried them, please share some info.

Talk smtp to gmail with openssl s_client

Here are the basic commands to talk smtp to gmail.

We will send email from sender@gmail.com to recepient@gmail.com
The gmail password of sender@gmail.com is "my secret password".

Preparation:

To authenticate, we need our user/pass in base64 format:
base64("sender@gmail.com") = c2VuZGVyQGdtYWlsLmNvbQ0K
base64("my secret password") = bXkgc2VjcmV0IHBhc3N3b3Jk

To get the base64 encoded string, google "base64 online encoder" and click on any of the online encoder/decoders.

If you're using Gmail's two-step authentication

Go to https://security.google.com/settings/security/apppasswords and get a one-time password.

Ending the DATA of the email.

To end the DATA part, we need to press dot (".") and then Enter (which should send CRLF).

Important NOTE: I'm on a macbook, and the terminal client sends LF when I press enter. When I want to send CRLF, I press Ctrl+V, Enter. If you don't know what I'm talking about, after the dot (".") if it doesn't work with dot and Enter, press [dot, Ctrl+V, Enter].

The commands

We will use S_client which is like telnet, but supports SSL (encrypted telnet). You will need OpenSSL for that purpose.

[mihail@arch ~]# openssl s_client -connect smtp.gmail.com:587 -starttls smtp
[a lot of text will be printed - ssl info. For simplicity ignore it.]
---
250 SMTPUTF8
auth login
334 VXNlcm5hbWU6
c2VuZGVyQGdtYWlsLmNvbQ0K
334 UGFzc3dvcmQ6
bXkgc2VjcmV0IHBhc3N3b3Jk
235 2.7.0 Accepted
helo
250 mx.google.com at your service
mail from:<sender@gmail.com>
250 2.1.0 OK dc8smxxxxwib.7 - gsmtp
rcpt to:<recepient@gmail.com>
250 2.1.5 OK dc8smxxxxwib.7 - gsmtp
data
354 Go ahead dc8smxxxxwib.7 - gsmtp
from:<sender@gmail.com>
to:<recepient@gmail.com>
subject:manual smtp with gmail
some text as the body of the email
more lines of text

.
250 2.0.0 OK 1414600919 dc8smxxxxwib.7 - gsmtp
quit
221 2.0.0 closing connection dc8smxxxxwib.7 - gsmtp
read:errno=0
[mihail@arch ~]#

 

 

Update1:

If you want to use SSL 465, the command is:
#openssl s_client -connect smtp.gmail.com:465 -tls1

(here you need to start with HELO, and then AUTH LOGIN - I don't know why)

Update2:

Also, some accounts fail with:

3073894076:error:1409E0E5:SSL routines:SSL3_WRITE_BYTES:ssl handshake failure:s3_pkt.c:598:

I don't know why. It seems like it wants to fall back to ssl3.

Simple and light bootable flash to change NT password (I tested on Windows Vista)

I recently had to fix a changed Administrator password on a Windows Vista (most probably by malware, it disabled all admin users and changed passwords).

One can only use the Ubuntu Desktop installer with chntpw, but it is large and it's pointless to use it for this task. I found a very simple distro to use for the same task.

There is one tool I would not recommend: Ophcrack. It is supposed to "hack" (actually an unoptimized bruteforcer). I couldn't find a way to just "change" the pass. It didn't have chntpw bundled.

Here's a link: http://pogostick.net/~pnh/ntpasswd/ (it is very small, AFAIR only 3-10mb).

Home servers - good at first, but have to go

For quite sometime I've been hosting some stuff at home. It's quite simple - a second hand small-form-factor machine from Compaq, home internet and a router/switch.

Compaq machines are were small and very very quite, they just don't break at all. The first machine that I used for hosting had 7 (seven!) fans some of which we very noisy. No keyboard and no monitor. A small ups and it's good to go.

Hosting stuff at home is extremely cheap - internet that would have been bought anyway. The machines are also very cheap - my blog was running on a Pentium III, 800 Mhz and the machine was running Windows Server 2003 with the blog being .NET application. It's a little load on the electricity bill but that's hardly a show stopper.

A good wi-fi router would do the job of connecting all machines and provide internet for everything in the house. Most new routers have enough juice to support a linux - I've mentioned OpenWRT. If one wants to go crazy one would move a *nix machine in front of the router (two ethernet cards needed), because the router could easily go down.

At first it's very interesting. There are different OSes to configure, building an internal dns, dhcp, port forwards, certificates, sharing. There are a lot of things to learn.

But at some point all this becomes cumbersome - different OSes to configure, building an internal dns, dhcp, port forwards, certificates, sharing. Every single thing takes so much time and so much effort. And the bad thing is that this is a hobby, I'm not a systems administrator.

When I wanted to learn something new there was always twice the effort - once for administration (installing, configuring...), twice for the thing itself. Having stuff hosted somewhere else one does not care about all that. Just click install, or ftp and copy and that's it.

Sometime ago I decided to put a stop to all that. Hosting is so cheap now and the most valued thing (this blog) was the first to go to some distant server. It took me a couple of days but it worked. The old and frankly very stupid personal site I had got killed - after all simplifying is good right?

I've had so many computers, at some point I've owned 4 to 5 machines. I don't think that's necessary. I intend to get rid of all of them and have a nice 802.11 N router with an optical internet and ups.

So it was fun and I learned a lot but it has to end.