DEEPAK RAJA D: linux

Showing posts with label linux. Show all posts

Wednesday, July 15, 2015

Entropy in Linux CentOS 6.5 / RHEL 6.5 @ Virtualization TYPE1 and TYPE2 (Performance considerations)

In computing, entropy is the randomness collected by an operating system or application for use in cryptography or other uses that require random data. This randomness is often collected from hardware sources, either pre-existing ones such as mouse movements or specially provided randomness generators. (source wikipedia)

Current Available Entropy in Linux Kernel
# cat /proc/sys/kernel/random/entropy_avail
58

Having a very low entropy generally affects cryptography operations (SSL, HTTPS etc).

Generally entropy is linked to device errors from the hardware. In perfectly virtualized setup (like xen, KVM, virtual box) error stream from hardware is masked by the hypervisor layer. It is recommended to generate entropy by utility programs.

# yum install rng-tools.x86_64

[root@perf04 init.d]# cat /etc/sysconfig/rngd
# Add extra options here
EXTRAOPTIONS="-r /dev/urandom -o /dev/random -t 5"
[root@perf04 init.d]# /etc/init.d/rngd status
rngd is stopped
[root@perf04 init.d]# /etc/init.d/rngd stop
Stopping rngd: [FAILED]
[root@perf04 init.d]# /etc/init.d/rngd start
Starting rngd: [ OK ]

# cat /proc/sys/kernel/random/entropy_avail
2780

Friday, July 10, 2015

NFS Setup and Performance Tunning on RHEL 6.5 / CentOS 6.5

Install nfs in Server and Client

# yum install nfs-utils nfs-utils-lib nfs-common
# yum install portmap

At Server(192.168.1.1)

Edit /etc/exports file for which directory to export
# cat /etc/exports
/nfsfs 192.168.1.2(rw,sync,no_root_squash)

# chkconfig nfs on
# service rpcbind stop
# service nfs stop

# chkconfig nfs on
# service rpcbind start
# service nfs start

# exportfs -a

At Client(192.168.1.2)

# chkconfig nfs on
# service rpcbind stop
# service nfs stop

# chkconfig nfs on
# service rpcbind start
# service nfs start

# mkdir -p /nfsfs

# cat /etc/fstab
192.168.1.1:/nfsfs /nfsfs nfs auto,soft,noatime,nolock,bg,intr,tcp,actimeo=1800 0 0
# mount -a

# df -kh
192.168.1.1:/nfsfs 44G 5.7G 36G 14% /nfsfs

VIM Editor Tips and Tricks (Linux and UNIX)

To replace ctrl M charter from a file.

:%s/^M//g

To make the above ^M to appear inside VIM command press ctrl +v then ctrl + m.

To Make the file format to UNIX.

When you open the file VIM reports file format as — DOS or Unix:

To modify the file format to UNIX use the below commands inside VIM

:set filetype=unix

:set fileformat=unix

The file will be written back without carriage return (CR, ^M) characters.

Remember to save.

:wq

In case you have to overwrite by force

:wq!

X11 GUI via SSH in CentOS 6 / RHEL 6

Enable X11 forward in Putty in local machine.

Install Cygwin in local machine and Open Xlaunch. use the following settings. Click all Next and close.

Launch the Putty session and connect to remote linux machine.

[root@test ~]# echo $DISPLAY
localhost:10.0
[root@test ~]# xclock

Now gui Open a XWindow in your local machine.

CPU Clock Speed Monitoring in Linux

Shell command to Monitor CPU Clock speed in Linux

bash# watch grep \"cpu MHz\" /proc/cpuinfo

Every 2.0s: grep "cpu MHz" /proc/cpuinfo
cpu MHz : 2792.990
cpu MHz : 2792.990

Thursday, November 21, 2013

Safe Delete(Shred) DATA in HDD/ Partition in LINUX

What Happens when you issue remove command on File System?

Most File System just remove the vnode link from the DIR structure. So what happens to your DATA. You got is right, it just sits at the DISK, waiting for be overwritten or to be recovered by a Crash Recovery Software. The latter sounds good when you are actually using it, but if some one else uses it. You know the consequences.

How to Safe Delete/ Shred File?

Simple, use any of the above commands

sudo dd if=/dev/zero of=<Device Target>

sudo dd if=/dev/zero of=/dev/sda

To get the Device Target use the following command

fdisk -l

This command lists all the Disk Partition info with Disk targets.

You can also use this command, if you need to be so sure. This makes two passes in shredding.

sudo shred -vfz -n 1 <Device Target>

sudo shred -vfz -n 1 /dev/sda

You can also use the LiveBoot of gparted.iso(http://gparted.org/). Boot it via CD/USB. And issue the above command even in Windows/ NON-Linux x86 Machines.

Note : Be careful when playing with these commands. You may purge away useful data Partition/Disk.

Monday, May 20, 2013

STATIC IP CONFIGURATION IN UBUNTU / LINUX

# cat /etc/network/interfaces
-----------------------interface file--------------------------------
# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).

# The loopback network interface
auto lo
iface lo inet loopback

# The primary network interface
auto eth0
iface eth0 inet static
address 192.X.X.X
netmask 255.Y.Y.Y
broadcast 192.X.X.X
gateway 192.X.X.X
dns-nameservers 192.X.X.X

-----------------------interface file ends-----------------------------

# ifconfig eth0 down
# ifdown eth0
# service networking restart

# ifconfig eth0

route info
# route -n

dns-servers placeholder under /etc/network/interfaces updates <resolve.conf> info too.

# cat /etc/resolv.conf
nameserver 192.X.X.X

CPU Frequency Governors / Power Schemes in LINUX

This feature is available since 3.4 Kernel.
To see the current speed.
watch grep \"cpu MHz\" /proc/cpuinfo
cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_max_freq
cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_min_freq

CPU POWER SCHEMES

Performance - Full throttle CLOCK
Powersave - Run the cpu at the minimum frequency
Userspace - User Specified CLOCK
Ondemand - Dynamically switch between CPU(s) available if at 95% cpu load
Conservative - Dynamically switch between CPU(s) available if at 75% load

More Info
https://www.kernel.org/doc/Documentation/cpu-freq/governors.txt
https://wiki.archlinux.org/index.php/CPU_Frequency_Scaling

To See current power scheme, below is the command. To modify the current scheme just write the new scheme name to the each CPU scale_governor files.

# cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor
ondemand
# cat /sys/devices/system/cpu/cpu1/cpufreq/scaling_governor
ondemand

INSTALLING UBUNTU 13.04 64bit OS under UEFI (GUID Partition Table)

UEFI BIOS Setting

Fast Boot : DISABLED
Secure Boot : DISABLED/OTHER OS TYPE
CSM(Compatibility Support Module)
Boot Device Control : UEFI and LEGACY, UEFI First
Boot From Storage Device : UEFI and LEGACY, UEFI First

GTP Partitioning of the DISK

UEFI Mandates this GTP partition table. GPARTED tool makes this partition happen.
http://gparted.org
http://en.wikipedia.org/wiki/GUID_Partition_Table
Download ISO burn to CD/USB. Boot up and setup the Partition below vi GPARTED Application.
Before doing the partition, you have to format the disk with GTP partition table via GPARTED.

400MB System Partition FAT32 as per the UEFI and GUID Partition Table Specifications. This has to be in the start of the disk. Make sure to set the BOOTABLE flag = true
250GB NTFS
30GB EXT4 for Linux root
4GB LINUX-SWAP
Apply/Commit all the changes to the disk.

BOOTING IN UEFI Mode

Most Important : Make sure to Boot the UBUNTU 13.04 CD in UEFI CD Boot mode. There shall be plain boot mode for the same CD. Careful in your selection.

More Info @ https://help.ubuntu.com/community/UEFI

Now boot up the UBUNTU 13.04 CD in UEFI mode.

400MB System partition has to be mapped to EFI PARTITION as File system during the UBUNTU installation. Rest of the Partition have be mapped for UBUNTU installation(ie root and swap).

After the installation and reboot,you shall see this layout below. GRUB2 is installed under EFI PARTITION which is mounted as /boot/efi

# df -kh
Filesystem Size Used Avail Use% Mounted on
/dev/sda3 30G 904M 28G 4% /
/dev/sda1 400M 132K 400M 1% /boot/efi

SHOW PARTITION TABLE

# gdisk
GPT fdisk (gdisk) version 0.8.5
Type device filename, or press <Enter> to exit: /dev/sda
Partition table scan:
MBR: protective
BSD: not present
APM: not present
GPT: present
Found valid GPT with protective MBR; using GPT.

Command (? for help): p
Disk /dev/sda: 3907029168 sectors, 1.8 TiB
Logical sector size: 512 bytes
Disk identifier (GUID): 7FE7EE77-D71C-48B5-B2EF-BEDA1B4C4C7E
Partition table holds up to 128 entries
First usable sector is 34, last usable sector is 3907029134
Partitions will be aligned on 2048-sector boundaries
Total free space is 1213466733 sectors (578.6 GiB)

Number Start (sector) End (sector) Size Code Name
1 2048 821247 400.0 MiB EF00 EFI Partition
2 821248 525109247 250.0 GiB 0700
3 2097973248 2160887807 30.0 GiB 0700 Dom0 root
4 2160887808 2169276415 4.0 GiB 8200

Command (? for help): i
Partition number (1-4): 1
Partition GUID code: C12A7328-F81F-11D2-BA4B-00A0C93EC93B (EFI System)
Partition unique GUID: 97D85749-AEAF-4767-95D0-1AAA2905FA3C
First sector: 2048 (at 1024.0 KiB)
Last sector: 821247 (at 401.0 MiB)
Partition size: 819200 sectors (400.0 MiB)
Attribute flags: 0000000000000000
Partition name: 'EFI Partition'

ALL THE AVAILABLE PARTITION TYPES

Command (? for help): l
0700 Microsoft basic data 0c01 Microsoft reserved 2700 Windows RE
4200 Windows LDM data 4201 Windows LDM metadata 7501 IBM GPFS
7f00 ChromeOS kernel 7f01 ChromeOS root 7f02 ChromeOS reserved
8200 Linux swap 8300 Linux filesystem 8301 Linux reserved
8e00 Linux LVM a500 FreeBSD disklabel a501 FreeBSD boot
a502 FreeBSD swap a503 FreeBSD UFS a504 FreeBSD ZFS
a505 FreeBSD Vinum/RAID a580 Midnight BSD data a581 Midnight BSD boot
a582 Midnight BSD swap a583 Midnight BSD UFS a584 Midnight BSD ZFS
a585 Midnight BSD Vinum a800 Apple UFS a901 NetBSD swap
a902 NetBSD FFS a903 NetBSD LFS a904 NetBSD concatenated
a905 NetBSD encrypted a906 NetBSD RAID ab00 Apple boot
af00 Apple HFS/HFS+ af01 Apple RAID af02 Apple RAID offline
af03 Apple label af04 AppleTV recovery af05 Apple Core Storage
be00 Solaris boot bf00 Solaris root bf01 Solaris /usr & Mac Z
bf02 Solaris swap bf03 Solaris backup bf04 Solaris /var
bf05 Solaris /home bf06 Solaris alternate se bf07 Solaris Reserved 1
bf08 Solaris Reserved 2 bf09 Solaris Reserved 3 bf0a Solaris Reserved 4
bf0b Solaris Reserved 5 c001 HP-UX data c002 HP-UX service
ef00 EFI System ef01 MBR partition scheme ef02 BIOS boot partition
fd00 Linux RAID

Monday, August 9, 2010

Using Solaris 10 Obervability Tools

Solaris 10 provides a bunch of observability tools to observe the processes, memory, processor, IO etc. Observing the system metrics data on solaris 10 can be done vi, SMC (Solaris Management Console) and command line stat utilities(iostat, vmstat etc).

Ideal way to observe a process/application is to observe it memory footprints, IO, processor needs etc. This process looks easy for small scenarios. However when we have a huge piles of data for analysis to find a root cause of problems, we end up looking up in verbose information. To make our life easier, we can stick with graphs generated with the system metrics data obtained earlier.

We will be using the outputs generated from the stats utilities. we shall extract the relevant information for our analysis by a shell script code. Later using OpenOffice/Staroffice/MS office spread sheet to draw necessary graphs.

Using Command line stat utilities have some benefits like mining for precise information, automation along
with any test procedure. vmstat, iostat, mpstat are available by default with the solaris 10. Nic stat is available under http://blogs.sun.com/timc/entry/nicstat_the_solaris_and_linux

bash-3.00# iostat -xn 2 >iostat_xn.txt &

bash-3.00# mpstat 2 > mpstat.txt &

bash-3.00# mpstat -a 2 >mpstat_a.txt &

bash-3.00# vmstat 2 > vmstat.txt &

bash-3.00# nicstat 2 > nicstat.txt &

bash-3.00# ls

iostat_xn.txt mpstat.txt mpstat_a.txt nicstat.txt vmstat.txt

bash-3.00#

These stat commands output lot of information. However, we may be interesed in quite a few according

to the current context. To filter out the required information I have come up with a shell script name extract.sh'. This script understands the output formats from vmstat, iostat -xn, mpstat, nicstat etc.. and filters out the unwanted information.

Download extract.sh

VMSTAT

bash-3.00# more vmstat.txt

kthr memory page disk faults cpu

r b w swap free re mf pi po fr de sr m0 m2 m2 s0 in sy cs us sy id

0 0 0 6904784 3290616 0 3 1 0 0 0 0 0 0 0 0 544 756 464 2 1 97

0 1 0 6279816 2622256 15 273 1660 12 12 0 0 0 0 0 225 1279 4322 1678 18 4 78

0 0 0 6279568 2618408 1 133 1955 0 0 0 0 0 0 0 261 1302 3847 1760 17 4 79

0 1 0 6279568 2614304 108 174 4376 0 0 0 0 0 0 0 199 1218 3740 1330 20 4 76

1 0 0 6279560 2606776 115 150 5324 0 0 0 0 0 0 0 113 1132 3868 1194 37 4 59

0 0 0 6279512 2597400 121 212 5824 0 0 0 0 0 0 0 96 1156 5323 1564 38 6 56

0 0 0 6279408 2590792 165 245 3679 0 0 0 0 0 0 0 141 1240 5056 1710 32 6 63

0 0 0 6267080 2569328 52 68 770 4 4 0 0 0 0 0 21 1063 4742 1227 51 3 46

1 1 0 6266968 2565368 140 228 3052 20 20 0 0 0 0 0 172 1234 6156 1875 35 6 59

0 1 0 6266824 2553392 178 246 2308 84 40 0 0 0 0 0 132 1266 7518 1984 48 5 47

4 0 0 6263736 2540344 185 286 2278 0 0 0 0 0 0 0 81 1557 10847 3006 58 7 35

2 0 0 6258344 2524432 166 172 2484 84 84 0 0 0 0 0 129 1560 11966 2182 76 8 17

4 0 0 6232992 2511904 139 381 1703 60 60 0 0 0 0 0 128 1251 4913 1303 63 5 32

bash-3.00#

bash-3.00# /deepak/extract.sh

Usage : extract <STAT NAME> <regular exp> <i/p file> <o/p file>

bash-3.00#

bash-3.00# /deepak/extract.sh vm " " vmstat.txt vmstat.out

extract exec begin

extract exec done

bash-3.00# more vmstat.out

VM STAT begin

usr sys idle

2 1 97

18 4 78

17 4 79

20 4 76

37 4 59

38 6 56

32 6 63

51 3 46

35 6 59

48 5 47

58 7 35

76 8 17

63 5 32

VM STAT done

bash-3.00#

Launch Openoffice or Staroffice calc

Create a new Spreadsheet. Copy paste the values from .out file.

Select the all the values. Go to Insert menu -> Chart.

IOSTAT

bash-3.00# more iostat_xn.txt

extended device
statistics

r/s w/s kr/s kw/s wait actv
wsvc_t asvc_t %w %b device

0.1 0.1 1.1 4.1 0.0 0.0
0.0 7.6 0 0 c0t0d0

0.0 0.0 0.0 0.0 0.0 0.0
0.0 14.4 0 0 c0t1d0

0.0 0.0 0.0 0.0 0.0 0.0
0.0 0.2 0 0 c1t2d0

bash-3.00# /deepak/extract.sh io_xn "c0t0d0" iostat_xn.txt iostat_xn.out

extract exec begin

io_xn

extract exec done

bash-3.00# more iostat_xn.out

IO STAT XN begin

IO device : c0t0d0

kr/s(c0t0d0) kw/s(c0t0d0)
asvc_t(c0t0d0) %b(c0t0d0) device(c0t0d0)

1.1 4.1 7.6 0 c0t0d0

1398.3 514.3 3.5 75 c0t0d0

1739.4 0.0 3.1 81 c0t0d0

4176.6 1023.9 3.8 72 c0t0d0

5524.9 0.0 4.7 45 c0t0d0

5496.2 1072.3 5.1 39 c0t0d0

3774.6 335.7 3.2 41 c0t0d0

823.8 28.2 4.0 10 c0t0d0

2619.4 570.4 3.4 48 c0t0d0

2300.9 660.2 4.1 37 c0t0d0

1961.3 533.7 6.3 35 c0t0d0

2533.4 920.2 6.4 40 c0t0d0

1495.6 115.8 3.6 36 c0t0d0

done

bash-3.00#

MPSTAT

a. Individual processor reporting

bash-3.00# more mpstat.txt

CPU minf mjf xcal intr ithr csw icsw
migr smtx srw syscl usr sys wt idl

0 1 0 3 428 147 328 3
9 2 0 429 2 2 0 96

1 2 0 2 116 54 136 2
10 2 0 327 2 1 0 98

CPU minf mjf xcal intr ithr csw icsw
migr smtx srw syscl usr sys wt idl

0 226 51 210 773 418 874 77
59 8 1 2168 18 4 0 78

1 224 138 363 517 296 841 92
60 10 0 2634 20 6 0 75

bash-3.00# /deepak/extract.sh mp "0 1" mpstat.txt mpstat.out

extract exec begin

extract exec done

bash-3.00# more mpstat.out

MP STAT XN begin

CPU : 0

minf(0) mjf(0) xcal(0) intr(0) csw(0)
icsw(0) migr(0) smtx(0) srw(0) syscl(0) usr(0) sys(0) idl(0)

1 0 3 428 147 328 9 2 0 429 2 2 96

226 51 210 773 418 874 59 8 1 2168 18 4
78

done

CPU : 1

minf(1) mjf(1) xcal(1) intr(1) csw(1)
icsw(1) migr(1) smtx(1) srw(1) syscl(1) usr(1) sys(1) idl(1)

2 0 2 116 54 136 10 2 0 327 2 1 98

224 138 363 517 296 841 60 10 0 2634 20
6 75

done

bash-3.00#

b. Processor set reporting

bash-3.00# /deepak/extract.sh mp_a "0" mpstat_a.txt mpstat_a.out

extract exec begin

mp_a

extract exec done

bash-3.00# more mpstat_a.out

MP STAT XN begin

SET : 0

minf(0) mjf(0) xcal(0) intr(0) csw(0)
icsw(0) migr(0) smtx(0) srw(0) syscl(0) usr(0) sys(0) idl(0) sze(0)

3 0 5 544 201 464 19 4 0 756 2 1 97 2

358 190 560 1284 710 1686 118 18 1 4482
19 4 77 2

122 226 547 1307 734 1744 108 32 2 3654
17 4 79 2

done

bash-3.00#

NIC STAT

bash-3.00# more nicstat.txt

Time Int rKB/s wKB/s
rPk/s wPk/s rAvs wAvs %Util Sat

12:40:30 lo0 0.00 0.00
0.03 0.03 0.00 0.00 0.00 0.00

12:40:30 bge0 4.15 0.10
3.87 1.49 1099.2 69.54 0.03 0.00

Time Int rKB/s wKB/s
rPk/s wPk/s rAvs wAvs %Util Sat

12:40:32 lo0 0.00 0.00
0.00 0.00 0.00 0.00 0.00 0.00

12:40:32 bge0 209.8 4.85
149.0 76.51 1441.3 64.90 1.76 0.00

bash-3.00#

bash-3.00# /deepak/extract.sh nic "bge0" nicstat.txt nicstat.out

extract exec begin

nic

extract exec done

bash-3.00# more nicstat.out

NIC STAT begin

NIC : bge0

Int(bge0) rKB/s(bge0) wKB/s(bge0)

bge0 4.15 0.10

bge0 209.8 4.85

bge0 235.3 5.16

bge0 324.1 7.28

bge0 217.3 4.88

bge0 230.0 5.03

bge0 221.8 5.21

bge0 216.1 5.06

bge0 221.2 5.04

bge0 219.6 5.10

bge0 246.9 7.14

bge0 352.6 8.32

done

bash-3.00#