Posts

List python libraries use for data science

Image
Data Processing and Model Deployment NumPy NumPy is a library used for numerical computing in Python. It provides fast array computations and is used for performing mathematical operations on arrays and matrices. Website:  https://numpy.org/ Pandas Pandas is a library used for data manipulation and analysis. It is used for reading, writing, and analyzing data in a tabular format. Website:  https://pandas.pydata.org/ Scikit-learn Scikit-learn is a machine learning library used for building predictive models. It provides a wide range of tools for data preprocessing, feature extraction, and model selection. Website:  https://scikit-learn.org/ TensorFlow TensorFlow is a popular library for building machine learning models, especially neural networks. It is widely used for deep learning tasks and provides tools for building, training, and evaluating models. Website:  https://www.tensorflow.org/ Keras Keras is a high-level neural networks API, written in Python and capable of running on top

Data engineering ecosystem

Image
𝗗𝗮𝘁𝗮 𝗘𝗻𝗴𝗶𝗻𝗲𝗲𝗿𝗶𝗻𝗴 is a role that requires building systems to process data efficiently and to model the data to power analytics.These solutions can range from batch data pipelines real-time processing services that integrate with core product features. 𝗗𝗮𝘁𝗮 𝗘𝗻𝗴𝗶𝗻𝗲𝗲𝗿𝗶𝗻𝗴 Involves a rich understanding of large distributed systems on which data solutions rely. it makes it possible to take vast amounts of data, translate it into insights, and focus on the production readiness of data and things like formats, resilience, scaling, and security. 𝗣𝗿𝗼𝗰𝗲𝘀𝘀𝗶𝗻𝗴 𝗕𝗮𝘁𝗰𝗵 - processing a large volume of data all at once. Hadoop Spark 𝗦𝘁𝗿𝗲𝗮𝗺- processing of a continuous stream of data immediately as it generates. Flink Spark Streaming 𝗗𝗮𝘁𝗮𝗯𝗮𝘀𝗲𝘀: SQL and NoSQL their schemas are predefined or dynamic, how they scale, the type of data they include and whether they are more fit for multi-row transactions or unstructured data. 𝗦QL Oracle mysql 𝗡𝗼𝘀𝗾

Top python libraries and frameworks

Image
  Top python libraries and frameworks.

Redis explained

Image

How to restart iBus on Fedora to use GNOME 3

Image
ibus on Fedora to use Gnome 3 not provide to restart it. So, if you want to restart ibus, you may try doing the below steps: Step 1: To kill current ibus daemon pkill -o ibus-daemon Step 2:  To start a new ibus daemon in Gnome 3, run this command in “Enter a command” tool by Alt+F2 /usr/bin/ibus-daemon --replace --xim --panel disable End.

Convert Unix timestamp to a date string in Linux

  With GNU's   date   you can do: date - d "@$TIMESTAMP" # date -d @0 Wed Dec 31 19 : 00 : 00 EST 1969 On OS X, use  date -r . date - r "$TIMESTAMP" Alternatively, use  strftime() . It's not available directly from the shell, but you can access it via gawk. The  %c  specifier displays the timestamp in a locale-dependent manner. echo "$TIMESTAMP" | gawk '{print strftime("%c", $0)}' # echo 0 | gawk '{print strftime("%c", $0)}' Wed 31 Dec 1969 07 : 00 : 00 PM EST

Frequently used OpenSSL Commands

Image
General OpenSSL Commands Generate a new private key and CSR (Unix) openssl req -out CSR.csr -pubkey -new -keyout privateKey.key Generate a new private key and CSR (Windows) openssl req -out CSR.csr -pubkey -new -keyout privateKey.key -config .shareopenssl.cmf Generate a CSR for an existing private key openssl req -out CSR.csr -key privateKey.key -new Generate a CSR based on an existing certificate openssl x509 -x509toreq -in MYCRT.crt -out CSR.csr -signkey privateKey.key Generate a self-signed certificate openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout privateKey.key -out certificate.crt Remove a password from a private key openssl rsa -in privateKey.pem -out newPrivateKey.pem Check the CSR, Private Key or Certificate using OpenSSL Check a CSR openssl req -text -noout -verify -in CSR.csr Check a private key openssl rsa -in privateKey.key -check Check a certificate openssl x509 -in certificate.crt -text -noout Check a PKCS#12 file (.pf

Golang - For loop

Three-component loop sum := 0 for i := 1; i < 5; i++ { sum += i } fmt.Println(sum) // 10 (1+2+3+4) This version of the Go for loop works just as in C/Java/JavaScript. The init statement,  i := 0 , runs. The condition,  i < 5 , is evaluated. If it’s true, the loop body executes, otherwise the loop terminates. The post statement,  i++ , executes. Back to step 2. The scope of  i  is limited to the loop. While loop If the init and post statements are omitted, the Go  for  loop behaves like a C/Java/JavaScript while loop: power := 1 for power < 5 { power *= 2 } fmt.Println(power) // 8 (1*2*2*2) The condition,  i < 5 , is evaluated. If it’s true, the loop body executes, otherwise the loop terminates. Back to step 1. Infinite loop By also leaving out the condition, you get an infinite loop. sum := 0 for { sum++ // repeated forever } fmt.Println(sum) // unreachable For each loop Looping over elements in slices, arrays, maps, channels and

Golang - Writing data to file

package main import ( "bufio" "fmt" "os" ) func main() { fileHandle, _ := os.Create("output.txt") writer := bufio.NewWriter(fileHandle) defer fileHandle.Close() fmt.Fprintln(writer, "String I want to write") writer.Flush() }

Golang - Read a file line by line

import ( "bufio" "os" ) func FileToLines ( filePath string ) ( lines [] string , err error ) { f , err := os . Open ( filePath ) if err != nil { return } defer f . Close () scanner := bufio . NewScanner ( f ) for scanner . Scan () { lines = append ( lines , scanner . Text ()) } err = scanner . Err () return }

Scanning a Linux Server for Malware and Virus by ClamAV

Image
The ClamAV antivirus software and how to use it to protect your server or desktop (Linux/Windows platform. It's also worth noting that ClamAV doesn't behave like a Windows antivirus. It doesn't hog up RAM or run in the background all the time. It also doesn't have all of the extra bells and whistles. It scans for viruses, and that's about all. 1. SETUP - Install on Fedora/CentOS/Redhat yum install clamav - Install on Debian/Ubuntu apt-get install clamav - Compile from source code   + Download:  http://www.clamav.net/downloads/production/clamav-0.100.0.tar.gz   + Compile: tar xvf  clamav-0.100.0.tar.gz cd clamav-0.100.0/ ./configure --prefix=/opt/clamav make sudo make install All pages will be placed in /opt/clamav 2. CONFIGURING All Clamav's configuration in /opt/clamav/etc (contains 2 files: clamav.conf and freshclam.conf) - clamd.conf (The configuration file save all Clamav's configrations) # BASIC LogFile /var/log/clamd.log

Building webserver with golang

Image
How to build a simple webserver with Go Code: package main import ( "fmt" "net/http" ) func main () {     http. HandleFunc ("/", func (w http.ResponseWriter, r *http.Request) { fmt. Fprintf (w, " Golang webserver. ")     })     http. HandleFunc ("/hello", func(w http.ResponseWriter, r *http.Request) { fmt. Fprintf (w, " Hello World! ")     })     fs := http. FileServer (http.Dir("static/"))     http. Handle ("/static/", http. StripPrefix ("/static/", fs))     http. ListenAndServe (":3000", nil) }

Using mutt email client to check mail

Image
What is Mutt? Mutt is a command line based Email client. It’s a very useful and powerful tool to send and read mails from command line in Unix based systems. Mutt also supports POP and IMAP protocols for receiving mails. It opens with a coloured interface to send Email which makes it user friendly to send emails from command line. Features Its very Easy to install and configure. Allows us to send emails with attachments from the command line. It also has the features to add BCC (Blind carbon copy) and CC (Carbon copy) while sending mails. It allows message threading. It provides us the facility of mailing lists. It also support so many mailbox formats like maildir, mbox, MH and MMDF. Supports at least 20 languages. It also support DSN (Delivery Status Notification). 1. INSTALLATION - Ubuntu/Debian apt-get install mutt - Fedora/CentOS yum install mutt 2. CONFIGURATION 2.1. Configuring for an account mail Create mkdir -p ~/.mutt/cache/headers mkdir ~/.mutt/cac