Questions tagged [large-data]
24 questions
1
vote
1 answer
Offsite backup solutions for 40TB of data/35 Million Files on single drive
I've got a server with 40TB of data and 35 million files. While the server itself has raid and all that jazz, I'm concerned about what would happen should something physically destroy the server (fire, lightning, etc).
The system is designed to run…
Jeff
- 21
- 1
1
vote
3 answers
Sync Large File Shares
What is the best way to sync large sets of files between to geographically separate locations so that both locations have the same file?
We have a constantly changing 500gig file share in Location1 with an avg rate of change of (+/- 5 gigs an hour…
l30
1
vote
6 answers
SQL Server 2000 tables
We currently have an SQL Server 2000 database with one table containing data for multiple users. The data is keyed by memberid which is an integer field. The table has a clustered index on memberid.
The table is now about 200 million rows. Indexing…
user40766
- 161
- 3
1
vote
1 answer
Varnish crashes on bigger files
Overall, I serve very small files. Think about images and small videos. Caching these with Varnish is like a breeze and doesn't give me any issues.
The problem I am having, is when I am downloading a 6 GB file. When doing so, I see the memory being…
P.T.
- 113
- 6
1
vote
1 answer
Multiple disks acting as one filesystem, while still allowing each disk to be accessed alone
I have a fairly large dataset (~160TB) that need to be delivered to a client every so often. This dataset consists of fairly large files, usually between 2Gb and 20Gb each. They exist on a BeeGFS filesisystem running on a RAID cluster with a total…
Jarmund
- 535
- 2
- 6
- 17
0
votes
0 answers
Tips on Copying 3 Database files nearing 250 GB?
We are upgrading from SQL Workgroup 2005 to SQL Standard 2012. We have plenty of databases of smaller sizes that copied over with straight windows file copy in a reasonable time frame. We have one database that is in 3 files, 53 GB, 93 GB and…
Eric
- 11
- 2
0
votes
0 answers
High RAM usage when providing large file on apache
I'm using Apache 2.4 on AlmaLinux 8. When I publicate a hundreds GB file and if someone starts downloading it, Apache RAM usage (especially VIRT) becomes bigger and bigger, starts swapping and ultimately, boom.
Why this happens, and is there any way…
NON
- 13
- 2
0
votes
1 answer
GCSFuse v0.33.2 - Large objects (11GB) upload to GCSBucket Fails
Receiving below errors when we try to upload 11 GB file to bucket; also, filesystem is going into "????????" as we can't list the objects available in the bucket.
This requires umount and mount back again to see the objects in the bucket. But, still…
-1
votes
2 answers
MySQL replcating huge changing data
Hi I have a master MySQL server with 950 gb of data and slave has stopped for some reason and there is a difference of 80gb of data.
so I started to setup new slave .
Since the data is changing and multiple databases dumping, copying and restoring…
Damn devil
- 9
- 2