Redis on CentoOS
:-
1.
pre-req (yum install make gcc wget)
3.
tar –xvf <the file, downloaded via wget/curl
from #1>, make, make test, make install :- yum install <package>
4.
ensure that file limits are adequate. Adjust the
parameters from the below files.
a. /etc/security/limits.conf
(optional)
b. /etc/sysctl.conf
i. sysctl
vm.overcommit_memory=1 (or else redis-server will warn you on startup)
ii. sysctl
-w fs.file-max=100000
iii. the
above changes will last only for the session, for the permanent change, edit
the file
5.
Add redis-server to chkconfig
a. move the executables from the directory where
the make install was run to /etc/init.d
6.
Or in lieu of 1 through 4 above steps use
yum install redis (you need to have the correct EPEL; http://dl.fedoraproject.org/pub/epel/,
select based on the architecture, etc.)
7.
Assign a password in redis.conf
8.
Start redis either through service start redis
or redis-server (inside redis-stable/src in the same directory where the make
install was fired.
9.
Ensure that redis is running
a. ps
–ef | grep redis
b. netstat
–nap | grep redis (ensure that 6379 which is the default port or any port if
you have changed the port in the redis.conf) (as root/privileged user)
c. ss
-nlp|grep redis (as root/privileged user)
d. redis-cli
ping
Redis on Azure
:-
Create a service. I used 2.5G capacity. Collect the key
(password), check off the SSL (in production you shouldn’t but I did deliberately
because the test on my localhost was conducted not over SSL, thus removed the
additional overhead of encrypt/decrypt), port.
Test using Java
:-
1.
pre-requisite:
a. java
b. maven
(optional)
c. eclipse
(optional) or any ide
2.
Approach :
a. using
maven or
b. download
jar file and create a Java class file in your favorite editor and use CLI (javac
and java)
3.
Pick a client from redios.io. (I used Jedis)
4.
Insure that Jedis java client library is compatible
with the Jedis server
5.
Maven artifact for jedis (http://mvnrepository.com/artifact/redis.clients/jedis/)
6.
Test via a java program.
Jedis jedis = new Jedis("localhost", 6379);
jedis.auth("password");
jedis.set("myid", "my id's value");
System.out.println("Output:" + jedis.get("myid"));
jedis.close();
A (quick and dirty)
performance test :-
You should use distributed multi machine test launch pad.
Ensure that the client (test) machines have sufficient RAM, appropriate
bandwidth, no other substantial programs running on them, etc. Ensure that the
redis-server is optimized with sufficient memory, less latency IOPs
(persistence SSDs, SAN, NAS (not preferred over SAN), etc.)…you get the point…there
are so many variables that can help improve the performance.
The test program executes sequentially write to the
cache.
Java client code for the test:
static String value = "qwertyuiopasdfghjkl;zxcvbnm,./qwertyuiopasdfghjkl;zxcvbnm,./qwertyuiopasdfghjkl;zxcvbnm,./qwertyuiopasdfghjkl;zxcvbnm,./qwertyuiopasdfghjkl;zxcvbnm,./qwertyuiopasdfghjkl;zxcvbnm,./qwertyuiopasdfghjkl;zxcvbnm,./qwertyuiopasdfghjkl;zxcvbnm,./qwertyuiopasdfghjkl;zxcvbnm,./qwertyuiopasdfghjkl;zxcvbnm,./qwertyuiopasdfghjkl;zxcvbnm,./qwertyuiopasdfghjkl;zxcvbnm,./qwertyuiopasdfghjkl;zxcvbnm,./qwertyuiopasdfghjkl;zxcvbnm,./qwertyuiopasdfghjkl;zxcvbnm,./qwertyuiopasdfghjkl;zxcvbnm,./qwertyuiopasdfghjkl;zxcvbnm,./qwertyuiopasdfghjkl;zxcvbnm,./qwertyuiopasdfghjkl;zxcvbnm,./qwertyuiopasdfghjkl;zxcvbnm,./qwertyuiopasdfghjkl;zxcvbnm,./qwertyuiopasdfghjkl;zxcvbnm,./qwertyuiopasdfghjkl;zxcvbnm,./qwertyuiopasdfghjkl;zxcvbnm,./qwertyuiopasdfghjkl;zxcvbnm,./qwertyuiopasdfghjkl;zxcvbnm,./qwertyuiopasdfghjkl;zxcvbnm,./qwertyuiopasdfghjkl;zxcvbnm,./qwertyuiopasdfghjkl;zxcvbnm,./qwertyuiopasdfghjkl;zxcvbnm,./qwertyuiopasdfghjkl;zxcvbnm,./qwertyuiopasdfghjkl;zxcvbnm,./qwertyuiopasdfghjkl;zxcvbnm,./qwertyuiopasdfghjkl;zxcvbnm,./124"; //1K
long outerbound = 10000;
String host = "localhost";
int port = 6379;
String password = null;
if (args.length >= 1) {
outerbound = Long.parseLong(args[0]);
}
if (args.length >= 2) {
host = args[1];
}
if (args.length >= 3) {
port = Integer.parseInt(args[2]);
}
if (args.length >= 4) {
password = args[3];
}
System.out.println("total sets:" + outerbound + ",host:" + host + ",port:" + port + ",password:"+password);
// JedisPool pool = new JedisPool(new
JedisPoolConfig(), "localhost", 6379,
Protocol.DEFAULT_TIMEOUT, "password");
Jedis jedis = new Jedis(host, port);
if (password != null) {
jedis.auth(password);
}
long i = 0;
long startTime = System.currentTimeMillis();
try {
// Jedis jedis =
pool.getResource();
for (i=0; i < outerbound; i++) {
jedis.set(""+i, value);
}
} catch (Exception e) {
e.printStackTrace();
long stopTime = System.currentTimeMillis();
long elapsedTime = stopTime - startTime;
System.out.println("error at: " + i + ",
time taken: " + elapsedTime);
throw e;
} finally {
// pool.destroy();
jedis.close();
}
long stopTime = System.currentTimeMillis();
long elapsedTime = stopTime - startTime;
System.out.println("out: " + elapsedTime);
Troublshoot:
·
make MALLOC=libc (or
yum install jemalloc-devel)
[zmalloc.h:50:31:
error: jemalloc/jemalloc.h: No such file or directory
zmalloc.h:55:2:
error: #error "Newer version of jemalloc required"]
·
update-alternatives --display java | grep 1.8 (find the proper version of Java to
use)
Test in a VM in
Azure:
VM: A1. (hence no SSDs or premium storage; for production
load you might want premium storage for the redis database persistence file.)
Cmd: starttime=`echo $(($(date +%s%N)/1000000))`;/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.45-28.b13.el6_6.x86_64/jre/bin/java
-cp .:jedis-2.7.2.jar App
100 localhost;finishtime=`echo
$(($(date +%s%N)/1000000))`; echo 'starttime:' $starttime, 'endtime:' $endtime;
diff=`expr $starttime - $endtime`; echo 'diff:' $diff
·
Name of the java program.
·
Number
of 1024 char messages to be written to the cache.
·
Redis server host
·
%s :- seconds since 1970-01-01 00:00:00 UTC, %N
:- nano seconds
Stats:
Redis running in A1:
|
Num of 1K messages
|
Execution time from within jvm in milli
|
Comment
|
Execution time including creation & destruction of
jvm in milli
|
|
100
|
146
|
|
953846
|
|
1000
|
311
|
|
981507
|
|
10000
|
1697
|
|
995653
|
|
100000
|
9959
|
|
1008997
|
|
1000000
|
51536 (error) 513714
after successive writes
|
redis.clients.jedis.exceptions.JedisConnectionException:
java.net.SocketTimeoutException: Read timed out
[Because A1
reached its IOPs limit. For production use Premium storage (DS series at the
moment)]
|
1901596
|
Redis on Azure C2 (2.5GB): Client run from an A1
standard VM. Please note of the VM n/w bandwidth (though MSFT doesn’t publish
the n/w bandwidth in/out…but higher the VM the better n/w i/o.)
|
Num of 1K messages
|
Execution time from within jvm in milli
|
Comment
|
Execution time including creation & destruction of
jvm in milli
|
|
100
|
97
|
|
7293642
|
|
1000
|
667
|
|
6873256
|
|
10000
|
6407
|
|
7242799
|
|
100000
|
57571
|
|
7340372
|
|
1000000
|
575820
|
|
7443979
|
|
10000000
|
1033785 (error),
after successive 1828914 writes
|
redis.clients.jedis.exceptions.JedisDataException: OOM
command not allowed when used memory > 'maxmemory'.
[Ran out of
memory on Azure Redis]
|
8124634
|
No comments:
Post a Comment