Oct
28
awarded Popular Question
Jan
12
awarded Tumbleweed
Jan
5
awarded Scholar
Jan
5
accepted Chinese text rendering strangely after upgrade to 12.04
Jan
5
asked Ubuntu 13.10 networking doesn't work - wired and wireless
Oct
2
awarded Scholar
Oct
2
accepted 3D KNN index in PostGIS
Aug
18
comment Caching large objects in a python Flask/Gevent web service
@MarkusUnterwaditzer yep, I ended up going with redis. The difficulty with NLP though is that for most tasks you need the entire corpus loaded in memory, generally in some kind of specialized data structure. If you only need the corpus for a particular task sometimes it's possible to work around this (like I ended up doing), but I don't know of any datastore that would support the general case.
Aug
18
comment Caching large objects in a python Flask/Gevent web service
Thanks, this is what I was looking for. Didn't realize there was a global namespace I could use. However, I ended up going with redis and Flask-Cache after figuring out a way to split the corpus files into much more digestible chunks. It's super fast, and it's also now possible to update the corpus without any downtime.
Aug
18
accepted Caching large objects in a python Flask/Gevent web service
Aug
16
asked Caching large objects in a python Flask/Gevent web service
Aug
1
accepted Trouble using a lock with multiprocessing.Pool: pickling error
Jul
31
comment Trouble using a lock with multiprocessing.Pool: pickling error
I'm on Linux, sorry I forgot to add that!
Jul
31
asked Trouble using a lock with multiprocessing.Pool: pickling error
Jun
20
awarded Student
Jun
19
asked 3D KNN index in PostGIS
May
15
answered Best database management system for nested sorting?
May
15
comment Using sparse matrices/online learning in Naive Bayes (Python, scikit)
It looks like you're dealing with "tweet-length" documents here, have you seen libshorttext yet? I just started using it to do classification on a corpus of ~10million tweet sized documents, and it's super fast and accurate (I'm getting 80-90% accuracy with 6 categories and a training set of about 400 documents). And it's written in Python/C as a bonus! EDIT: and, I just realized this thread is almost a year old
May
15
answered Database that supports three-dimensional spatial indexing
May
15
asked Database that supports three-dimensional spatial indexing
1 2