[nSLUG] BLOB's and Large File Storage

George N. White III gnwiii at gmail.com
Mon Jun 1 20:03:51 ADT 2015


On Mon, Jun 1, 2015 at 2:51 PM, chris thompson <ct8ball at gmail.com> wrote:

>
> Happy Monday,
>
> Large File storage(Pictures). Is it of any benefit to store such files in
> a DB via BLOB? or alternately what in your opinion is the best way to store
> and share these on a local network? file-system play any part here? I'm
> very interested to hear the feedback from you folks as I'm trying to
> architect a mechanism for large file versioning if you will.... aka svn/cvs
> for large files, I know there are a few that exist, but I believe a
> ground-up approach needs to be taken to maximize the effectiveness of any
> solution.
>
> Try not to get duck feet out there.
>

Depends on how the files will be used.  I should mention hdf5 or netcdf4
formats with OpenDAP clientsand servers.  This supports discovery and
processing applications (Matlab and many others).

At work we are redesigning a legacy workflow that used externally
compressed hdf4 files with one based on netcdf4 with internal compression.
With the old workflow, users had to download the compressed files and
decompress for use, even if they wanted only a small subset of a file.  The
new workflow should allow users to load a variety of subsets directly into
applications.  The catch seems to be managing the metadata, but it is hard
to imagine that it could be worse than the old system where users often
needed lengthy email exchanges to get the metadata in a form they could use.

-- 
George N. White III <aa056 at chebucto.ns.ca>
Head of St. Margarets Bay, Nova Scotia
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://nslug.ns.ca/pipermail/nslug/attachments/20150601/bbd1bc21/attachment.html>


More information about the nSLUG mailing list