Posts

Showing posts from August, 2020

Azure Function App with Azure Map

Image
Photo by Andrew Neel on Pexels.com In this blog, we showed how to expose two Azure Map API via Azure Function App . The source code is at this G it repo . We can expose more API following the same pattern. All the information on development and deploying application are in the README.md file. Let's run through it. 1. Setup 1.1 Create Resource Group, Azure Map and Function App Create Azure Map Create Function App When creating Function App, please have the followings Runtime stack = Python Version=3.8 (or a version that matches your Python version) and add an Application Setting under Configuration Section. This allows the function app to make calls to Azure Map REST API.     SUBSCRIPTION_ID=<the primary/secondary key of Azure map> Add Application Setting You should have the followings in your Resource Group once you are done Apps in Resource Group 1.2 Git clone On your laptop, Git clone the repository git clone https://github.com/dennisseah/azure-fn-map.git 1.3 Setup Py

User Delegation SAS Token for Blog Storage

Image
  Photo from  Kaique Rocha on Pexels.com In my previous blog, Azure Blob Storage: SAS Token , we talked about using Azure Blob Storage Account key to generate Shared Access Signature ( SAS ) token. In this blog, we are going to show how to generate a User Delegation SAS token.  I created a Service Principal (you can create an Application ID instead if you wish to) and assigned  Storage Blob Data Contributor to it . This role assignment is required for this form of token to work. 1. Setup 1.1 Export Environment Parameters STORAGE_ACCOUNT_URL=<storage account URL> STORAGE_CONTAINER_NAME=<storage container name> STORAGE_BLOB_NAME=<storage blob name> AZURE_CLIENT_ID=<service principal or app id> AZURE_CLIENT_SECRET=<service principal or app secret> AZURE_TENANT_ID=<tenant id> 1.2 Install Python package pip install azure-identity pip install azure-storage-blob 2. Code Copy the code below to a file named main.py and execute it accordingly. python m

Azure Databricks Notebook in Azure ML pipeline

Image
  Photo by Tim Mossholder on Pexels.com Machine Learning pipelines are essential to automate machine learning workflows. It chains a sequence of data processing steps together to complete ML solutions. There are many workflow engines such as mlflow (a open source project), KubeFlow (another open source project), and in Microsoft , we have Azure ML pipeline . In this blog, we are doing to illustrate how to execute a Azure Databrick Notebook in Azure ML pipeline . 1. Sample Use Case One example of "embedding" a  Azure Databrick Notebook  in  Azure ML pipeline is the need to process massive data. Azure ML Pipeline The diagram above shows a simple ML pipeline where we have data preparation and training steps. Data is read from an  Azure Blob Storage  and size of the required data is not massive. Then, we may have another data source which is massive. We want to use Azure Databricks to help us to do ETL (Extract-Transform-Load) task to generate data for this ML pipeline. T

Azure Blob Storage: SAS Token

Image
  Photo by Max DeRoin on Pexels.com A shared access signature (SAS) provides secure delegated access to resources in your storage account without compromising the security of your data. With a SAS, you have granular control over how a client can access your data. You can control what resources the client may access, what permissions they have on those resources, and how long the SAS is valid, among other parameters. - Official Microsoft Docs . In this blog, we discussed about SAS  token for Azure Blob Storage . We talked about why this form of token is useful, the different kind of tokens, and code snippet to generate this token. We also briefly talked about generating this token via Azure Portal UI . Why and when do we generate a SAS token? In some cases, we need a  subject (can be a client, agent, or a person) to perform some tasks on Azure Blob Storage on our behalf. The task can be reading a blob in a blob container, or listing blob containers from the root of the storage accoun

Azure Batch Compute, Azure ML Service and Azure Databricks

Image
Photo by Manuel Geissinger on Pexels.com   Cloud Computing is ubiquitous as it is fueling for Digital Transformations . Companies are exploiting cloud platforms and blue-prints to jumpstart the executions of their  Digital Transformations strategies. Among other things, Cloud Computing offers On-demand availability Scalability Pay-per-use model comprehensive identity, authentication and authorization models logging and metric services secure connections to data sources In this blog, we look at Azure Batch Compute , Azure Machine Learning Service , and Azure Databricks Compute  platforms. Azure provides these different computing platforms to assist customers in getting their solutions up and running with minimal efforts. Fundamentally, they are the same because they provide computing resources. However, they are fabricated for different use cases therefore they have different concepts, and tool sets. Azure Batch Compute This is a lightweight and generic platform for parallel process

job interviews for software engineer positions

Image
  Photo by Startup Stock Photos on  Pexels.com After replying to this  tweet  from  Heba , I have a strong urge to write this blog because I feel that job interviews for software engineer positions can be improved. I am sharing some of my experiences, and observations. Puzzle In almost every interview, candidates are required to solve programming puzzles, and write code for them. This part of the interview process is typically the most difficult one. In order to do well in this area, candidates can engage professional coaches and instructors. Some people spend many hours/weeks/months on solving problems listed in online programming platforms such  leetcode . Coaching and training are needed because these puzzles are tricky and they require a lot of thinking. During the interviews, the candidates are given limited time to find good solutions for these puzzles. Hence, it can be extremely stressful, and they do not perform well. In my opinion, the ability to solve these puzzles does not t

numbers this week: Aug 3, 2020

Image
  Photo by Lukas on  Pexels.com These are some of interesting numbers surface this week. Number of COVID19 infected people in the United States of America The number of infected people in US has exceeded  5 million . The daily infection rate is above  60,000 , and the good news is that it is declining. Number of people died is over  162,000 . These information is from  COVID-19 Dashboard by the Center for Systems Science and Engineering (CSSE) at Johns Hopkins University (JHU) . The  5 million  jumped out at me because the population of  Singapore  is about  5.85 million . This means that almost the entire population in  Singapore  is infected based on this number (in US) alone. Evidently, there shall be significant health issues down the road because  COVID-19  viruses damages major organs like lungs, kidneys and gut. Apple' s market cap Apple ‘s market cap is nearly  $2 trillion , and it has overtook  Saudi Aramco  ( Saudi Arabia’s state-owned oil company ) as the company with hi

the world is not fair

Image
Photo by Matheus Bertelli on  Pexels.com  “the world is not fair”, “this is not fair”, … We heard these kind of claims many times. They are true for sure. Why are people frustrated about it when we should try to accept it? We are going to suffer more if we try to fight against it. Hence, the wise way to deal with it is to accept it and figure out how to alleviate the situations. There are people who do not shoes, and we have people who have too many pairs of shoes. There are people who have little to wear, and we have people who packed and stored their clothings in boxes. Some parents have disabled children, and lucky ones has healthy children. Some people are smarter than others. People in some countries have access and equal opportunities to education, and many do not. The  COVID19  pandemic statistic clearly shows that there are health disparities, example  Why People of Color Are Suffering More from COVID-19  and  Bill Gates Urges U.S. to Help Poorer Countries Get Vaccines . And, t

book: the culture map

Image
Photo by fauxels on  Pexels.com Have you read a book that every chapter resonates with your past experiences? I have because I have just finished reading  the Culture Map:  Breaking Through the Invisible Boundaries of Global Business , by  Prof. Meyer . I am fortunate to have a hardcover copy of the book and I am going to keep it. Typically, I give my books away to the public libraries or friends after reading it. But I am going to keep this one for future references. A few weeks ago, my manager introduced this book to us because we have a team of software engineers from different parts of the world. We migrated to  the United States , and we spent most of our educations and early careers in our countries. We have  6  people in the team and almost everyone from different countries. When I was little, my mother told me that children have  “eyes, ears, and no mouth” , we were supposed to observe, listen, and not speak (unless someone gave us the permissions) when adult were talking. This