Questions tagged [redis]

Redis is an open source (BSD licensed), in-memory data structure store, used as a database, cache and message broker. It supports data structures such as strings, hashes, lists, sets, sorted sets with range queries, bitmaps, hyperloglogs, geospatial indexes with radius queries and streams. It also provides pub-sub capabilities. Use this tag for questions related to Redis and in-memory system.

Redis

Redis is a BSD-licensed, advanced key-value store which works in-memory but provides disk persistence. It is often referred to as a data structure server since keys can contain as values different data structures:

  • Strings, which are binary safe and up to 512 MB in size.

  • Lists, offering O(1) push/pop/trim/length operations regardless of the number of elements contained inside the list. Lists also provide blocking operations (pop-style commands that block if there are no elements in the list), so Redis lists are often used in order to implement background jobs and other kinds of queues. There are very popular libraries like Resque and Sidekiq using Redis as a backend.

  • Hashes are field-value maps like in most programming languages. They are useful in order to represent objects and are very memory efficient for a small number of fields, yet very scalable supporting up to 2.14 billion fields per hash.

  • Sets are unordered collections of elements and are useful in order to add, remove, and test elements for existence in constant-time. Small sets of integers are extremely space efficient, and but sets scale up to 2.14 billion elements per set. It is possible to ask for random elements inside sets which is very useful. See SPOP and SRANDMEMBER for more information.

  • Sorted sets are very useful data structures where collections or elements are ordered by a floating point number called score. The data structure offers a set of very powerful operations running in logarithmic time: it is possible to add and remove elements, increment the score of elements, get ranges by rank and by score, given an element get its position (rank) or score, and so forth. A notable application is leader boards involving million of users: there are companies using Redis sorted sets in order to implement leader boards of popular games such as Facebook games.

  • Geo sets are sorted sets in which elements' scores are used for storing locations - longitude and latitude - as geohash-encoded values. Once stored in this fashion, the elements can be queried by their distance from an arbitrary position with the GEORADIUS command. Geospatial indexes are used by any location-based application and service, including: social networks, navigation & commuting assistance and fleet management.

  • Counters are not exactly a type per se, but actually operations you can use with strings that represent integers. For example, the command INCR mykey will automatically create a key with the string value "1" if the key does not exist. The next call will modify the value of the string into "2", and so forth. You can increment and decrement by floats or by any amount. Values are in the range of a signed 64-bit number even when using Redis on 32-bit architectures.

  • Bit operations, like counters, operate in strings in a different way. The user is basically able to treat the string as an array of bits, doing very memory-efficient operations. For example, if you have ten million users and want to store a Boolean value for every user, you'll need just a bit more than 1 MB of memory! Because of the rich set of bitwise commands you can: count the number of set bits with BITCOUNT; perform bitwise AND, OR, XOR, and NOT between bitmaps using BITOP; find the first bit clear or set in a given range with BITPOS; and so forth.

  • Bit fields are strings that, similarly to bit operations, are treated as an array of bits. These allow referencing integers of varying types (unsigned or signed 1-bit to 64-bits and 63-bits, respectively) by offset or position. Each such bit field can be read, written or incremented and supports several overflow modes via the use of the BITFIELD command.

  • HyperLogLog is a probabilistic data structure that efficiently (in terms of computational and memory complexity) addresses the count-distinct problem. The Redis implementation of HLL requires only 12KB for each counter and exhibits a standard error of 0.81%. HLLs can be added with items, merged and counted using the PFADD, PFMERGE and PFCOUNT commands, respectively (the PF prefix of the commands is in honor of Phillipe Flajolet, HyperLogLog's inventor).

  • Streams, that are structures that provide an abstraction of log-like append-only data. Messages in the stream are added by producers with the XADD command, and the processing of these is done by consumers with the XREAD. Streams also support the concept of Consumer Groups via the XREADGROUP for simple and efficient scaling.

  • Modules, that are basically just dynamically-loaded server-side libraries, can developed and used by anyone and everyone for extending the core Redis platform with anything from custom data types (e.g. a Bloom Filter) to full-fledged servers (e.g. a search engine). Modules are supported as of v4.

To get started quickly, try Redis directly inside your browser, read this quick intro to Redis data types, or watch a great presentation by Peter Cooper.

Features as a data store

While Redis is an in-memory system, it offers a lot of features of a data store.

  • Tunable on-disk persistence with a point-in-time snapshotting persistence, or an Append Only File with tunable fsync policy.
  • Asynchronous replication.
  • Redis is also a very fast Pub/Sub server.
  • An API to configure Redis at runtime and automatically rewrite the configuration file.
  • Automatic failover and monitoring via Redis Sentinel.
  • Shared-nothing clustering is available from v3.

It has an impressive ecosystem of client libraries for all the mainstream and elite programming languages.

Community

The Redis community is big and willing to help.

Persistency

There are two options for persistency in Redis:

  • RDB (Redis Database File): This option takes snapshot from database periodically.
  • AOF (Append Only File): Logs every write operation and reconstructs dataset at startup.

RDB is faster than AOF but loses created data after the latest snapshot.

Support

Support for Redis is provided by the following companies:

Certification

Redis has a Professional Certification program at no cost! There are three prerequisite courses that must be successfully completed before enrolling in the Developer Certification Program:

  1. Introduction to Redis Data Structures
  2. Redis Streams
  3. Any other elective Redis University class of your choice.

The Redis Certified Developer exam is a timed, 90-minute multiple-choice test. You can schedule your exam at any time and take it from any location, including your own home. You can learn more about the Redis Certified Developer Program from the official certification page.

Related tags

24955 questions
8
votes
2 answers

How to restore a redis backup in a redis container?

I have a redis container. I would like to back it up and reimport the backup on another machine in another redis container. I followed theses steps: # Create the original redis container docker run --name redis -d redis:3.0.3 redis-server…
poiuytrez
  • 21,330
  • 35
  • 113
  • 172
8
votes
2 answers

Install gcc on AWS EC2 without using yum?

I am trying to install Redis on EC2 but it needs gcc. When I tried gcc was not installed. Then I tried to install gcc manually and got the following error: configure: error: could not find a working compiler, see config.log for details How can I…
akshay hundia
  • 249
  • 1
  • 5
  • 12
8
votes
3 answers

Redis Pub/Sub Ack/Nack

Is there a concept of acknowledgements in Redis Pub/Sub? For example, when using RabbitMQ, I can have two workers running on separate machines and when I publish a message to the queue, only one of the workers will ack/nack it and process the…
Richard Knop
  • 81,041
  • 149
  • 392
  • 552
8
votes
4 answers

Celery error : result.get times out

I've installed Celery and I'm trying to test it with the Celery First Steps Doc. I tried using both Redis and RabbitMQ as brokers and backends, but I can't get the result with : result.get(timeout = 10) Each time, I get this error : Traceback…
MG1992
  • 537
  • 1
  • 8
  • 16
8
votes
1 answer

What is the score data type in redis sorted sets?

So I've taken a look at the redis docs as well as the question here about the redis internal data structures, but without going through the redis source I can't seem to get an answer as to what the data type is for the "score" of a sorted list. The…
longofest
  • 616
  • 5
  • 16
8
votes
2 answers

How to sort a redis hash by values in keys

is there a good way in redis to get keys in a hash sorted by values? I've looked at the documentation and haven't found a straightforward way. Also could someone please explain how sorting is achieved in redis, and what this documentation is trying…
Deepak Mishra
  • 297
  • 1
  • 5
  • 15
8
votes
1 answer

Redis PUB/SUB: how to ignore own messages?

The idea is: I have N WCF services which connected and subscribed to the same Redis message channel. These services use this channel to exchange messages to sync some caches and other data. How each service can ignore its own messages? I.e. how to…
ZedZip
  • 5,794
  • 15
  • 66
  • 119
8
votes
2 answers

Is exists check required before calling StringSet method of StrackExchange.Redis

I am using the StackExchange.Redis 1.0.450 nuget in C#. I have code like below which checks if a keyexists in redis before adding it - if (!Cache.KeyExists(fKey)) { Cache.StringSet(fKey, Serialize(data)); } where Cache is Database object I was…
Vishal Thakur
  • 95
  • 1
  • 9
8
votes
2 answers

How to configure ElastiCache Redis for Spring Data Access

I am trying to setup ElastiCache to use with a Java Application. I have based my setup based on this documentation: https://docs.aws.amazon.com/AmazonElastiCache/latest/UserGuide/BestPractices.html The EC2 instance where the Java (8) App runs is in…
asaraff2
  • 81
  • 1
  • 4
8
votes
2 answers

How to configure timeout of JedisConnectionFactory based on Spring Boot framework?

I'm using Spring Boot and I'm confused how to configure the timeout to connect Redis. Currently, my configurations are: application.yml: spring.redis.host: myhost spring.redis.port: 6379 spring.redis.pool.max-idle: 8 spring.redis.pool.min-idle:…
terry
  • 341
  • 2
  • 6
  • 13
8
votes
3 answers

Distributed Task Queue Based on Sets as a Data Structure instead of Lists

I'm wondering if there's a way to set up RabbitMQ or Redis to work with Celery so that when I send a task to the queue, it doesn't go into a list of tasks, but rather into a Set of tasks keyed based on the payload of my task, in order to avoid…
HostedMetrics.com
  • 3,525
  • 3
  • 26
  • 31
8
votes
3 answers

Celery: access all previous results in a chain

So basically I have quite a complex workflow, which looks similar to this: >>> res = (add.si(2, 2) | add.s(4) | add.s(8))() >>> res.get() 16 Afterwards it's rather trivial for me to walk up the result chain and collect all individual…
WhatIsName
  • 2,294
  • 4
  • 24
  • 36
8
votes
1 answer

How to format ServiceStack Redis connection string

How can I format the below Redis connection string: Connection string: myIP,keepAlive=180,ConnectRetry=30,ConnectTimeout=5000 I started writing a unit test but keep getting a input string was not in correct format error message [TestFixtureSetUp] …
user1526912
  • 15,818
  • 14
  • 57
  • 92
8
votes
2 answers

Multiple-get in one go - Redis

How can we use lua scripting to implement multi-get. Let's say if I've set name_last to Beckham and name_first to David. What should the lua script be in order to get both name_last and name_first in one go? I can implement something like: eval…
user_ab
  • 225
  • 1
  • 3
  • 6
8
votes
0 answers

Is there a way to achieve Autofailover and autodiscovery of redis nodes with JedisCluster?

I used Jedis Java client for Redis and it is fantastic. I used separately the features for clustering (JedisCluster) and for High-Availability (JedisSentinelPool). Both implementations work like a charm alone, however it doesn't seem to be a way to…
1vand1ng0
  • 1,203
  • 1
  • 16
  • 18