Monday, November 16, 2020

Tableau setting up

 Tableau setting up

 

Just a few particulars as I get started using Tableau Public:

 

1. The data for the x-axis should be put in the Column shelf, while the data for the y-axis should go in the Rows shelf. Contrary to the tutorial videos, I prefer not to just drag and drop icons on the main canvas and trust it to do the right thing.

2. Cntl or Shift select multiple measures to go in as Rows that share a common Column by dragging onto the y-axis tic mark area until you see a double short vertical bars. The legend is auto-generated and can be copied with a right mouse click context popup menu.

3.  After saving/uploading the visualization to the Tableau server, look for the "Metadata" link in the lower right corner of the webpage to see all the worksheets and dashboards listed by title. Click on the one you want. By default, Tableau always opens the Viz on the server to the last item.

4. When working in a Dashboard, look in the upper left hand corner for the "Range" and the drop down menu to the right of it. Choose "automatic" to fill your screen, which will hopefully also scale the viz to any other screen when people look at it on their own.

 5. Again in the Dashboard, click into the blank white space  subsection representing your row measure, then click on the "Color" mark button to get a simple menu of solid color choices for all of your bars in a bar chart. Otherwise, if you drag the row measure onto the button, the only color choices you'll see are all gradients.

6. To modify y-xis viewing parameters, right mouse click on the axis and then choose "Format ...". Changing the label size and font is done simultaneously with the tic mark size and font. Look for "Default" Font with a drop down menu.



 

 

 

 

 

 

 

 

 

 

Monday, November 2, 2020

SciPy Highlights

 The scipy module is based on the numpy array handlers and does a variety of statistical, parameter contour space searches for minima, special treatments of sparse matrices, interpolations and hypothesis testing.


Finding roots of an equation:


from scipy.optimize import root

from math import cos

from scipy import constants


# Note that the equation can remain generalized with no definite array for x


def eqn(x):

  return x**3 + cos(x)


myroot = root(eqn, (constants.pi/2)) # pi is a best initial guess at the root


print(myroot.x)


# results in:

# the solved for root is x. nfev is the number of evaluations of the function.

fjac: array([[-1.]])

     fun: array([1.11022302e-16])

 message: 'The solution converged.'

    nfev: 16

     qtf: array([-7.64299735e-12])

       r: array([-3.00853835])

  status: 1

 success: True

       x: array([-0.86547403])



We can also evaluate for the minimum of a function:

minimize(a, b, c, d, e)


a: the objective function

b: best initial guess

c: choice of evaluation method

d: callback - optional post-iteration function to use

e: dictionary of optional parameters


def eqn(x):

  return x**2 + x - 6


themin = minimize(eqn, 10, method='BFGS')


print(themin)


# results in:


fun: -6.249999999999978

 hess_inv: array([[0.49999999]])

      jac: array([2.98023224e-07])

  message: 'Optimization terminated successfully.'

     nfev: 8

      nit: 2

     njev: 4

   status: 0

  success: True

        x: array([-0.49999985])


If we try a cubic function though, the results are not usable.


def eqn(x):

  return x**3 + x - 6


themin = minimize(eqn, 750, method='BFGS')


print(themin)


# results in:

fun: array([-6.94927495e+08])
 hess_inv: array([[-0.00037628]])
      jac: array([2353680.])
  message: 'Desired error not necessarily achieved due to precision loss.'
     nfev: 276
      nit: 18
     njev: 132
   status: 2
  success: False
        x: array([-885.75370841])


GitHub Repositories

 

Python source file, MySQL Workbench model, EER Diagram and sample flat files and database dump file for the eBay collectible Stamp Grades Aspect Container ensemble statistics and Stamp World Region Category Container statistics.


https://github.com/DancingGuy/Stamp-Geographic-Statistics


https://github.com/DancingGuy/Stamp-Grade-Statistics



Tuesday, October 20, 2020

More SQL Commands

 More SQL Commands


SELECT DISTINCT column_name_1, column_name_2


is useful for grabbing all the unique fields elements and is often used with a


WHERE BETWEEN/LIKE/IN with operators >=, <=, <> and connectors OR/AND/NOT.


BETWEEN: inclusive of the start and endpoints

LIKE: pattern matching such as '%s' to find fields that end in the letter s.

IN: find members of a set. Think of it as multiple ORs.


SELECT * FROM table_name ORDER BY  column1, column2

first orders ASC by default on column1, then any ties are broken by sorting on column2. Note: you need to put the ASC or DESC after each column name.


INSERT INTO on less than all columns of a table will put a NULL into any column not specified with the command. So if you have a table with 10 columns, but only insert data into 4 of the columns, the 6 unassigned columns would automatically be filled with NULL (assuming the column is declared to be nullable in the design).


Test for NULL with IS NULL or IS NOT NULL.


UPDATE table_name SET column1=value1, column2=value2 WHERE conditions


This is a good way to change the timestamps that cross over the midnight boundary because your timezone is different than UTC.


SELECT MIN/MAX() AS variable_name FROM table_name

In MySQL Workbench, functions like MIN/MAX() will appear in gray.


produces a new variable.


COUNT/AVG/SUM() does not include any NULLs.


LIKE can take wildcards:


                                    % -> 0 more more characters in that position of the pattern

                                    _ -> a single placeholder for a character

                                        % ... % -> the ... in any position

Those wildcards also work against an INT.


ESCAPE: allows you to use special characters as literals in the pattern. For example, LIKE '543#%' means match 543% exactly, not wildcarded.


A powerful function in MySQL is REGEXP_LIKE() because it permits traditional regular expression syntax and pattern matching.


NOT IN is useful when followed by (SELECT ...)


(NOT) BETWEEN accepts numbers, text or dates and is often used with IN. If you are using it for dates, use a pair of # to bracket the date.


ALIAS only exists for the duration of the query.


UNION combines two or more SELECTs.


1. Each SELECT needs the same number of columns

2. All columns must be of the similar type to what it is lined up with.


UNION ALL will preserve multiplicity.


Resultant column names will inherit from the names of the columns from the first SELECT clause.


GROUP BY breaks down the various elements in a column into N subsets, gathered into any nth subset all together, for N distinct field values available.


HAVING is a pseudo-where to go with aggregation keywords such as GROUP BY, ORDER BY


EXISTS returns a boolean if any row survives the joins and conditions.


ANY and ALL are restricted to use with WHERE or HAVING only. ANY or ALL must be preceded by a compariosn operator such as > or <>.


SELECT INTO copies data into a new table. Supplementing with AS will provide new column names.


SELECT * INTO backupTable FROM old_table creates a copy in the same schema.


SELECT * INTO backupTable IN 'BackupSchema.mdb' FROM old_table creates a copy in a different schema.


TRICK: Create a new empty table with the same column design:


add WHERE 1=0 to force no matching.


INSERT INTO table_name(col names) SELECT another_table's columns FROM another_table


CASE

WHEN cond1 THEN res1

..........

ELSE default_res

END AS new column

Important: If all WHENs fail and there is no ELSE, then a NULL is returned. Otherwise it quits at the first true condition.

If there is no AS clause after the END keyword, the column is named in very verbose ugly way with all of the WHENs' result string concatened together.??!!

IFNULL() or COALESCE() allow you to substitute any non-null values for NULL in math calculations.


Stored Procedures:

CREATE PROCEDURE proc_name

AS

sql-commands

GO


Run it with:


EXEC proc_name


Procedures can receive parameters too.


use @param_name data_type


EXEC proc_name @param_name='...'


and add more parameters with commas


Comments are either -- or /* */


See the full list of SQL operator symbols at w3schools.com/sql/sql_operators.asp























Monday, October 19, 2020

eBay Finding API issues

 eBay Finding API issues


1. eBay on-line docs say that the string length for the primary and secondary category names are max at 30 characters but I found several names longer than 30.

2. A watchCount field is populated in the response object under the listingInfo grouping. This is not documented by eBay.

3. eBay has substituted the erstwhile CurrentPriceLowest Sort By Order argument with PricePlusShippingLowest.

4. Sometimes the API call hangs even after 3 retries. Change the timeout setting to be longer than the 20 second default in the Connection argument list.

5. A primary category number may have more than one primary category name. Fore example, primary category number 7921 is both "Collections, Lots" and "Collections/Mixtures".