Stack Overflow
Stack Overflow

Questions tagged [gfs]

GFS can mean either Google File System, or "GFS2" which is a cluster file system available in Linux.

GFS can mean either Google File System, or "GFS2" which is a cluster file system available in Linux.

48 questions
2
votes
1 answer

Google File System Application to Client Communication

From Google File System paper, Application communicates with GFS Client by sending (filename, byteRange) and then GFS Client converts that byteRange to ChunckIndex. From the document it is unclear to me Does byteRange represent begin and end of…
SlyBits
  • 31
  • 2
2
votes
2 answers

GFS/Hadoop master's storage capacity

I'm reading GFS paper but unable to understand one point, does master maintains 64kb of metadata for each replica of file(s) too? Say if master's memory is 8 gb and I store 1000 files of 1 kb each, how much memory it's going to take? if replication…
endurance
  • 31
  • 3
2
votes
1 answer

Is there any abstraction layer to work with GFS or HDFS?

The SQL and NOSQL databases are used by facebook 1.Whether it uses GFS or HDFS or BOTH or some other? 2.What are the different Abstraction application layer available to work on HDFS AND GFS ?? 3.I heard about HADOOP , How Map Reduce works ?…
JAVA Beginner
  • 389
  • 3
  • 15
2
votes
1 answer

Why Google File System divide files into Chunks with 64 MB？

Is there any reasons for dividing into 64 MB? And they divide each chunk as 64 KB blocks .Is there any reason for this?
Jarachanthan Ratnakumar
  • 656
  • 13
  • 20
2
votes
3 answers

Why Google File System divide files into Chunks？

In the paper The Google File System Section 2.3, it says: Files are divided into fixed-sizechunks. But don't tell why. What's the advantage for that?
Sayakiss
  • 6,878
  • 8
  • 61
  • 107
2
votes
1 answer

How does Google file document objects?

How does Google store and organize documents, such as documents in Google Docs? I'd like to ask which file system Google uses, but I know it uses GFS, a distributed file system for storing huge files, basically the huge databases containing, among…
azzurroverde
  • 567
  • 5
  • 18
1
vote
3 answers

What is atomicity in the context of filesystems?

According to webopedia it is when a processor both reads and writes to the bus...excluding other processors from accessing it. In databases, it means that a set of operations either all occur or none do, according to wikipedia In concurrent…
user656925
1
vote
0 answers

How google doc are stored on disk?

I know google docs are stored in google drive but what is the underlying storage used for google docs? My guesses are GFS: But GFS is not optimized for small files and a large number of small writes. BigTable: Each update is a new record. Every…
Paras Malik
  • 11
  • 2
1
vote
1 answer

how is a chunk server different from normal server?

I was reading the Google Bigtable paper and stumbled upon this word chunk server used in Google File System. So I wanted to know is there any difference in chunk server and simply a server. Thanks in advance!
Ravi Raja
  • 61
  • 9
1
vote
1 answer

Why do small files create hot spots in the Google File System?

I don't understand this from the Google File Systems Paper A small file consists of a small number of chunks, perhaps just one. The chunkservers storing those chunks may become hot spots if many clients are accessing the same file. What…
Abhirath Mahipal
  • 938
  • 1
  • 10
  • 21
1
vote
0 answers

How to read GRIB2 file with PHP

I would like to display weather forecast data coming from NOAA GFS data subset generated by http://nomads.ncep.noaa.gov/ filter. As far as I know, the output file is in GRIB2 format. Unfortunately I cannot find any PHP library, which will "give me"…
user3523426
  • 836
  • 3
  • 11
  • 26
1
vote
1 answer

Does Hadoop Distributed File System support any Update Operation as Google File System do?

I was reading paper published on Google File System and find out that GFS supports append and updates in arbitrary locations of an existing file. As far as I know that HDFS wont support update operation because it is meant for write once and read…
user1188611
  • 945
  • 2
  • 14
  • 38
1
vote
1 answer

Google File System Read consistency

Does google file system provide read consistency. I am confused because I know that the primary maintains write consistency in GFS. If a system provided write consistency is it not that it provides read consistency also? Thanks Manjit
manjit singh
  • 101
  • 2
  • 8
1
vote
0 answers

Is the HDFS sink in Flume using a "anti-pattern" with it's default config

looking at the HDFS sink default parameters in Apache Flume it seems that this will produce tons of very small files (1 kB rolls). From what I learned about GFS/HDFS is that blocksizes are 64MB and filesizes should rather be gigabytes to make sure…
pagid
  • 13,559
  • 11
  • 78
  • 104
0
votes
2 answers

What is the throughput of Google File System?

What is the throughput of Google File System?
user53670
1
2
3 4