Open Source Linux PostScript Software

ShellCheck is a GPLv3 tool that provides warnings and possible suggestions for bash/sh shell scripts. ShellCheck finds bugs in your shell scripts. You can cabal, apt, dnf, pkg or brew install it locally right now. ShellCheck highlights and clarifies typical beginner's syntax mistakes and issues that cause a shell to give a cryptic error message. It shows typical intermediate level semantic problems that cause a shell to behave in a abnormally and counter-intuitively. It can also discover ssubtle caveats, corner cases and pitfalls that may cause an user's working script to fail under probable future circumstances. ShellCheck.net is always synchronized to the latest git version, and is the simplest way to give ShellCheck a go.

pyspider is a powerful Spider(Web Crawler) system in Python. Components are connected by message queue. Every component, including message queue, is running in their own process/thread, and replaceable. That means, when process is slow, you can have many instances of processor and make full use of multiple CPUs, or deploy to multiple machines. This architecture makes pyspider really fast. benchmarking. Since pyspider has various components, you can just run pyspider to start a standalone and third service free instance. Or using MySQL or MongoDB and RabbitMQ to deploy a distributed crawl cluster. To deploy pyspider in product environment, running component in each process and store data in database service is more reliable and flexible. To deploy pyspider components in each single processes, you need at least one database service. pyspider now supports MySQL, MongoDB and PostgreSQL. You can choose one of them.