Commit Graph

834 Commits (con-cache-patch-test)

Author SHA1 Message Date
RobertJoonas 7ccbc5d432
fix typos (#2393) 4 years ago
Adam Rutkowski a44b8ff7f5
Improve test coverage (#2392)
Co-authored-by: ruslandoga <doga.ruslan@gmail.com>
4 years ago
Adam Rutkowski 6ba5e53574
Trim goals when creating and retrieving from the DB (#2382)
People are likely to enter (copy/paste) goals from external sources
which can lead to whitespace characters appended by accident.
That renders the goal unusable and hard to distinct visually.

Normally to fix up existing goals we would use a data migration,
but this should be good enough to check if the problem
with never appearing goals resurfaces.
4 years ago
Marko Saric 252cecbf25
Correct the spelling of JavaScript (#2380)
* Update README.md

* Update snippet.html.eex

* Update settings_general.html.eex

* Update site_controller_test.exs
4 years ago
Vinicius Brasil 73444ddfb6
Fix and test Google Analytics token refresh (#2381)
This commit fixes a bug where Google Analytics import tokens were not
being refreshed properly because the function was not returning the
expected tuple. Thanks to @aerosol we can nicely test this now.
4 years ago
Adam Rutkowski 8e75f2fc07
Separate GA/SC scopes (#2372)
* Use separate scopes for GA/SC integrations

* Update tests with google scope expectations
4 years ago
Adam Rutkowski d29597ed37
Fix random typos (#2375) 4 years ago
Vinicius Brasil c9945e09cf
Suggest enterprise plan when usage is greater than 10M pageviews (#2345) 4 years ago
Vinicius Brasil a86d9697ba
Clear stats_start_date after clearing GA stats (#2370)
This commit fixes a bug where users clearing one import and trying to
re-import would get invalid import date ranges. This was caused because
the stats_start_date field was not updated to nil after clearing
imported stats, as this field defines the end date of the import.
4 years ago
Adam Rutkowski 3ddfda94a1
Fix up Purge tests (#2364) 4 years ago
Vinicius Brasil e48851a9ed
Skip rows with empty dates from GA imports (#2359) 4 years ago
Adam Rutkowski 0fa6b688af
Google APIs integration improvements (#2358)
* Make TestUtils module available in all tests

* Add macros patching the application env in tests

Unfortunately a lot of existing functionality relies on
certain application env setup. This isn't ideal because
the app config is a shared state that prevents us from
running the tests in parallel.

Those macros encapsulate setting up new env for test purposes
and make sure the changes are reverted when the test finishes.

* Allow passing request opts to HTTPClient.post/4

We need this to swap custom request building in
Google Analytics import.

* Unify errors when listing sites

* React: propagate backend error messages if available

* React: catch API errors in Search Terms component

* Propagate google API errors on referrer drilldown

* Handle verified properties errors in SC settings

* Add missing tests for SC settings controller

* Unify errors for fetching search analytics queries (list stats)

* Unify errors refreshing Google Auth Token

* Test fetch_stats/3 errors and replace Double with Mox

* Fixup makrup

* s/class/className

* Simplify Search Terms display in case of errors

* Fix warnings
4 years ago
Vinicius Brasil 93bb62ef27
Remove inserts per second Google Analytics test assertion (#2334)
This commit removes a flaky test assertion from the GA suite. The
initial purpose of this test was to ensure we are not inserting to
Clickhouse more than 1 time per second during an import.

Although it is nice to have an assertion like this, the test is flaky
and fails sometimes. To get the total inserts during a time range, it
queries Clickhouse internal tables for insert commands, and this number
is not deterministic.

In a real-world scenario with Clickhouse, the app runs in multiple nodes
not aware of their neighbors' CH calls. In that case, Grafana provides
us visibility over CH, and these tests do not suffice.
4 years ago
Adam Rutkowski e3ca3b32db
Include tests for Captcha success/failure scenarios (#2344)
* Include tests for Captcha success/failure scenarios

* DRY
4 years ago
Vinicius Brasil c26e029d22
Increase flaky test timeout (#2330) 4 years ago
Adam Rutkowski 28cec9d939
Automatically decode JSON payloads from HTTP (#2329) 4 years ago
Vinicius Brasil fb05e447f9
Increase Google Analytics timeout and lower page size (#2326)
The Google Analytics report request may take some time, especially with big
imports with thousands of pages. To mitigate the issue this commit increases
the timeout to 60s and lowers the page size to 7,500 records per request.
4 years ago
Adam Rutkowski ae4ae5d0a0
Handle duplicate invitations gracefully (#2323) 4 years ago
Adam Rutkowski ec90a264b4
Improve user input validation (#2291)
* Make pagination params parsing ignore bad input

* Remove unused binding

* Don't crash on filter parse error

* Sanitize input date on internal stats API

* Revert Query module changes (ref 55645734)

* Implement simplistic input date validation in stats controller

* Mute bad request logging
4 years ago
RobertJoonas c0da024b23
Remove static tracker files (#2116)
* remove tracker files from git index

* generate tracker files on npm test

* generate tracker files for elixir tests/dev/CI

* update tracker/package-lock.json

* exclude npm run deploy from mix test + some docs
4 years ago
Vinicius Brasil 0733efa89e
Delete stats improvements (#2318)
* Move clear stats functions to Plausible.Purge

* Delete both native and imported stats when deleting a site

This commit moves the delete site function to the Plausible.Purge
module, and fixes a bug where deleted sites could leave dangling
imported stats.

* Clear sites.stats_start_date after clearing stats

This commit fixes a bug where resetting stats left an invalid state of
the stats_start_date field, used for GA imports, for example.
4 years ago
Adam Rutkowski 523b114682
Allow super-admin Stats API access for locked sites (#2308) 4 years ago
Vinicius Brasil beabef7269
Redirect user after upgrading only when subscription is created (#2285)
* Create API to check for an active subscription

* Redirect after upgrading only when subscription is created

* Change upgrade page copy

* Add timeout notice to upgrade success page
4 years ago
Adam Rutkowski bf2b6880c5
Block Stats API for locked sites (#2302)
* Extract Sites.locked? predicate

* Lock Stats API when dashboard is locked

* Tidy tests

* Don't pollute application env from tests

* Add changelog entry

* Revert "Add changelog entry"

This reverts commit 76346074f9.
4 years ago
Uku Taht 0bc762bfdc
Fix svg favicons (#2295)
* Overrides content-type for SVG favicons

* Organize favicon rendering

Make sure the placeholder icon is always requested from
/favicon/sources/placeholder

* Run prettier on site-switcher.js

* Yak Shave: upgrade Heroicons to 2.0

* Use HeroIcons instead of custom svg

* Update lib/plausible_web/plugs/favicon.ex

Co-authored-by: Adam Rutkowski <hq@mtod.org>

Co-authored-by: Adam Rutkowski <hq@mtod.org>
4 years ago
Uku Taht 12a513f3cd
Remove Mimic dependency (#2280) 4 years ago
Uku Taht 4096d88171
[Sentry APP-189] Fallback to current date in all time parsing (#2277)
* Fallback to current date in all time parsing

* Add changelog
4 years ago
Uku Taht 65f0ce3061
[Sentry APP-5B] Ensure form errors can be rendered on /settings (#2278)
* Ensure form errors can be rendered on /settings

* Changelog
4 years ago
Uku Taht d104abb53d
Add fallback for favicon (#2279)
* Add fallback for favicon

* Add Favicon tests

* Changelog

* Move placeholder icon to priv folder
4 years ago
Vinicius Brasil 7489290d11
International Domain Names (IDN) Support (#2034)
* Accept letters from non-Latin alphabets in domain names

* Replace static URLs with Router functions in settings_visibility

* Beautify dashboard URL in visibility tab

* Add IDN support to CHANGELOG
4 years ago
Vinicius Brasil c07361fbce
Increase GA import request receive_timeout to 30s (#2274)
This commit increases the receive_timeout from 15s to 30s for Google
Analytics import requests. Timeouts have been reported, and I could
replay some requests that took 15-20 seconds to complete when fetching
10,000 records in a single request.
4 years ago
RobertJoonas 155e274150
Stats API: allow escaping `|` literal character with `\|` when filtering (#2266)
* add separate module for filter parsing

* add tests for filter parser

* allow escaping pipe character in filter value

* add documentation and doctests

* do not remove escape chars from wildcard values

* changelog update

* change the parse_filters/1 function argument
4 years ago
Vinicius Brasil db03cc0ac6
Fix timezone list when time changes (#2268)
This commit fixes an exception when calling `Timex.Timezone.get/2` after
there has been a timezone change. Similar issues have already been reported
in Timex (bitwaker/timex#691, bitwalker/timex#555, bitwalker/timex#625).

This skips timezones from the list when they fail to fetch, and use
`Timex.Timezone.get/2` to get the current offset using `DateTime.utc_now/0`
instead of the `NaiveDateTime` default.
4 years ago
Vinicius Brasil 1adda42a75
Refresh Google Analytics token before import (#2254)
* Capture refresh and expires from GA callback

* Pass GA refresh token to import worker

* Refresh GA token before import
4 years ago
Uku Taht e849e03058
Fix favicons (#2257) 4 years ago
RobertJoonas 80e01fedd5
do not show invoices for free subscription (#2249)
* do not show invoices for free subscription

* use refute instead of negative assert

* changed to refute in the other test too
4 years ago
Vinicius Brasil 2b753cdcfd
Improve GA Imports error reporting (#2244)
* Return error tuple instead of exception when GA request fails

* Report Google Analytics failed request body to Sentry

* Improve Google Analytics function @doc

* fixup! Report Google Analytics failed request body to Sentry

* fixup! Improve Google Analytics function @doc
4 years ago
Uku Taht e16e357dd2
Fix shared link with bad auth (#2225)
* Render 404 when shared link cannot be found

* Add documentation for StatsController and shared link rendering

* Refactor shared_link/2 for more clarity

* Add changelog entry

* Use mermaid graph for sequence diagram

* Use more accurate return value in sequence diagram

* Refactor Ecto query to be more idiomatic

* Remove order dependence in test

* Restore backwards compatibility for older shared links

* Add changelog entry
4 years ago
Uku Taht 669091f2ef
Ignore unknown country in imported data (#2247) 4 years ago
Vinicius Brasil 40182f6d6e
Manually lock outgrown enterprise accounts (#2197)
* Rename enterprise_over_limit e-mail template

* Change suggested_plan/2 function to suggest enterprise plans

* Start grace period for enterprise accounts

* Create separate module for GracePeriod schema

* Enable manual locking of enterprise accounts

* Update outgrown account alert for enterprise accounts

* Document account locking feature

* fixup! Enable manual locking of enterprise accounts
4 years ago
Adam Rutkowski 75264f8f1c
Treat non-200 HTTP responses as errors (#2243)
* Tag non-200 HTTP responses as errors

Co-authored-by: RobertJoonas <robertjoonas16@gmail.com>

* Send get_invoices/1 errors to Sentry

Co-authored-by: RobertJoonas <robertjoonas16@gmail.com>

* Keep Google API module matching non-200 responses

Co-authored-by: RobertJoonas <robertjoonas16@gmail.com>

* Make sure HTTPClient.Error.t() doesn't appear in the UI

* Unify get_invoices/1 signature

Co-authored-by: RobertJoonas <robertjoonas16@gmail.com>

Co-authored-by: RobertJoonas <robertjoonas16@gmail.com>
4 years ago
Uku Taht 5d4918b66b
Allow admins to grant admin rights (#2216)
* Make sure admin can grant the admin role

* Use `site` and `current_user_role` from conn.assigns to avoid extra lookups

* Add some missing documentation

* Restrict owners in the `update_role` action

* Move @moduledoc to appropriate location

* Use Ecto.Enum to get role atoms

* Remove unused functionality

* Ensure that owners cannot lock themselves out

* Add question mark to `can_grant_role`

Co-authored-by: Adam Rutkowski <hq@mtod.org>

* Use strict short-circuit operator for booleans

Co-authored-by: Adam Rutkowski <hq@mtod.org>

Co-authored-by: Adam Rutkowski <hq@mtod.org>
4 years ago
Uku Taht b239f73a6d
Ignore unknown country code (#2223)
* Ignore unknown country code

* Add changelog entry
4 years ago
RobertJoonas 0194d57b38
add upper bound to limit parameter (#2226)
* add upper bound to limit parameter

* add more strict validation for limit parameter + 2 tests

* update changelog
4 years ago
Vinicius Brasil 178e34b9e7
Fix timezone input (#2222)
* Update Timex version from 3.7.7 to 3.7.8

* Generate timezone list from Tzdata

This commit fixes a bug where timezone changes weren't updating the
timezone list displayed when editing or creating a site.

Timezones were being pulled from a static list. This commit changes it
to generate the list from Tzdata, that uses a timezone database with
updated information on time changes. Additionally it adds more timezones
with aliases and links to the list.

Closes #1340

* Use timezone name from browser to recommend timezone

This commit matches the timezone name instead of offset to recommend a
timezone when creating a new site. The JavaScript Intl.DateTimeFormat
API is widely supported according to the link. In any case, if the
timezone fails to match by name, it fallbacks to the offset strategy.

https://caniuse.com/mdn-javascript_builtins_intl_datetimeformat_resolvedoptions_computed_timezone

Closes #904
4 years ago
Adam Rutkowski cc4d4bb8e0
Fixup error reporting (ref #2181) (#2204)
* Fixup error reporting (ref #2181)

Fixes two problems:

 - `extra` sent to Sentry is always expected to be a map
 - no `String.Chars` protocol is implemented for `Finch.Error` struct

* Use explicit paths in http client tests

* Make HTTP Client tests sync
4 years ago
Vinicius Brasil d31db86b49
List all Google Analytics views during import (#2184)
* List all Google Analytics views during import

This commit fixes a bug where different Google Analytics views with the
same name and URI were not shown. This was caused because GA views were
stored as a map, that naturally doesn't support duplicate keys.

This change updates the GA views list to display view IDs, making it
clearer to know what is being imported. The dropdown is now grouped by
website URL.

* Put Google Analytics API URLs in app env

* Add controller test to GA view list
4 years ago
Adam Rutkowski e2aa519c31
Test HTTPClient (#2185)
* Sort dependencies

* Add :bypass dependency

* Test HTTPClient

* Rename test module
4 years ago
Adam Rutkowski 9c23e886aa
Merge pull request #2174 from plausible/fix-flaky-test-assuming-order
Do not assume any order in browser data import test
4 years ago
Adam Rutkowski 79b4550bed Do not assume any order in browser data import test 4 years ago
Adam Rutkowski 2275cd8e22 Respond with 400 when no auth supplied to shared link request 4 years ago
Vinicius Brasil bc68235d04
Add fallback for GA reports without rows (#2121) 4 years ago
Vinicius Brasil a10d44a0d7
Refactor event struct creation function (#2098)
* Replace Ingestion.Request headers with user_agent

* Replace generic Ingestion.Request params with specific fields

* Refactor event building function into small functions

* Move Plausible.Ingestion to Plausible.Ingestion.Event

* Add option to override event fields while building

* Rename Ingestion.Request meta to props

* Replace UTM-specific fields with generic query_params

* Remove Map.from_struct/1 call from ingestion pipeline

* Remove stash options from ingestion
4 years ago
Vinicius Brasil 4d20c7ce70
Catch Google Search Console grant error (#2101)
* Remove invalid Jason.decode argument

Co-authored-by: Robert Joonas <robertjoonas16@gmail.com>

* Add custom message to Google invalid grant error

Co-authored-by: Robert Joonas <robertjoonas16@gmail.com>

* Test invalid_grant while refreshing Google token

Co-authored-by: Robert Joonas <robertjoonas16@gmail.com>

Co-authored-by: Robert Joonas <robertjoonas16@gmail.com>
4 years ago
Uku Taht a5757bc61e Allow ownership transfers 4 years ago
Vinicius Brasil 6b8ed12567
Fix Google Search Console API call (#2090)
This commit fixes a bug where fetching Google Search Console keywords
raised a FunctionClauseError. This was introduced in #2046. This commit
also adds test coverage.
4 years ago
ruslandoga ce461b5192 Auth updates 4 years ago
RobertJoonas a058cf6240
added a test and changed hashmode condition (#2082) 4 years ago
Vinicius Brasil 4b9032d822
Google Analytics Import Refactor (#2046)
* Create separate module for GA HTTP requests

* Fetch GA data entirely instead of monthly

* Add buffering to GA imports

* Change positional args to maps when serializing from GA

* Create Google Analytics VCR tests
4 years ago
Igor Kapkov 61fe38e0d4
Fix script path for k6 load test (#2068) 4 years ago
Vinicius Brasil b415ebe776
Fix geolocation subdivision pattern matching (#2063)
* Fix geolocation subdivision pattern matching

This commit fixes a bug where regions were not being saved. This was
caused because Geolix response was returning an additional
`:geolocation` map key. It also adds a test case for this.

Closes #2033

* Add geolocation database to .gitignore
4 years ago
Vinicius Brasil b5ea6ae3dc
Keep user filter when listing cities, countries, and regions stats (#2030)
This commit fixes a bug where location filters were filtering stats but
not the locations list. This was caused by a `Map.put/3` call that
overrides the user filter. This commit rollbacks 5b57143273
changes and removes the `Map.put/3` call.

Closes #1982
4 years ago
Sasha Fonseca db3a852f26
Fix missing 'pageviews' column from 'pages.csv' (#1988)
* Fix missing 'pageviews' column from 'pages.csv'

* Add CHANGELOG entry for Issue #1878 fix
4 years ago
Uku Taht 292a419473
Add feature flag to block event ingest (#1991)
Event ingest can be blocked using the flag `block_event_ingest`
4 years ago
Uku Taht 3e5695408a
Use new Session.CacheStore in favour of Session.Store (#1934)
* Remove Session.Store in favour of Session.CacheStore

* Add CHANGELOG entry

* Use appropriate enum function
5 years ago
Kurt McAlpine b92889aaf0
Add API to retrieve site by domain (#1942) 5 years ago
Uku Taht 102ff1885e Trim trailing whitespace from pathname 5 years ago
RobertJoonas 11654ddc07
Script extension additions (#1915)
* added data-include attribute to plausible.exclusions.js

* reorder extensions in filename when serving the plausible script

* fix formatting

* tweaks after review

* changelog update
5 years ago
Uku Taht 948de2b4c9 Mark more tests as async 5 years ago
Uku Taht cd8df23e5e Do not rely on insertion order in tests 5 years ago
Uku Taht 6f3399bb7c Fix annual notification loop 5 years ago
Uku Taht e4816e8f64 Update Elixir version 5 years ago
Uku Taht 02abbda06d Ignore nil custom prop as well 5 years ago
Uku Taht c92c548ca8 Do not error on empty string custom prop value 5 years ago
Uku Taht f18a211dcc
Ingest throughput improvement test setup (#1867)
* Add OTEL and test Cachex for sessions

* Move load test

* Start apps in the appropriate order
5 years ago
Uku Taht ce55dc4ae9
Retries Google Analytics API request if it returns empty rows (#1858)
* Retries Google Analytics API request if it returns empty rows

* Remove HTTPoison.Retry completely
5 years ago
Uku Taht 5e415c2420 Add entry_props back in 5 years ago
Uku Taht 8fb4f3f886 Revert entry props 5 years ago
RobertJoonas 40275b64d4
Pageview custom dimensions (#1816)
* added custom dimension filtering tests for pages

* first filter UI in place

* pages, entry pages and exit pages can be filtered by pageview props

* added tests for expected filtering behaviour

* fix dimension filter for sources + tests

* added is_not filtering functionality

* fixed formatting

* fixed admin_test

* added (none) as filter value + is_not filter type in UI

* added prefilling applied filter values and some UI tweaks

* added fetch options

* Make prop suggestions work with `props` filter

* Fix test

* Track login state internally

* Add CHANGELOG entry

Co-authored-by: Uku Taht <uku.taht@gmail.com>
5 years ago
Uku Taht 66aceda9e9 Fix main graph selection with imported data 5 years ago
Uku Taht 404a5d15a3 Fix tests 5 years ago
Uku Taht 05c210ad85 Enforce site limit 5 years ago
Uku Taht 3d231fc094 Redirect to correct change plan page for enterprise customers 5 years ago
Andrea Mazzarella 1128ff4bfa
Update the internal /sites api to paginate results (#1824)
* Update the internal /sites api to paginate results and adapts site-switcher to it

* Update the Changelog

* Format internal controller

* Remove the `+ Add Site` link from the site-switcher in the dashboard

* Change camel to snake case and replace imports with fully qualified calls

* Remove trailing comma from site-switcher
5 years ago
Uku Taht d2dc06ec96 Fix trial notifications 5 years ago
Vignesh Joglekar 3b97ecdc62
Adds Main Graph Metric Selection (#1364)
* First pass bringing in previous graph improvements, and comparsion context

* Swaps issue template to new issue form syntax

* Indentation update

* Indentation update?

* More indentation

* Intendation is hard

* Finalized indentation?

* Github indentation

* Missing fields

* Formatting changes

* Checkbox changes

* Uses new timeseries API, various UI improvements, descopes conversions, ToP from graphing

* Fixes Mobile UI Issues

* Improves point detection and display on hover

* Fixes & adds tests for updated main-graph API route

* Changelog

* Changes to better metric option declaration & minor UI/default fixes

* Fixes top stat tooltips showing unformatted numbers for special (non-rounded) top stats

* Formatting

* Fixes regression with dashed portion not stopping at present_index

* Removes comparison + lint

* Improves top stat active style

* Removes comparison tests

* Splits out tooltip and top stats

Still needs:
- Tests
- Potentially more cleanup

* Adds/moves tests for top stats

* Formatting

* Updates metric LS key, removes console log

* Various fixes + cleanup

* Makes tooltip position & style more consistent

* Fixes test (returns import status on both main graph & top stats)

* Fixes interaction with month dateFormatter

* Fixes edge case tooltip behavior

It was simpler than I thought :/

* Make the entire top stat clickable

* Minor UI improvements

* Fixes another tooltip visibility edge case + cleans up boolean algebra

Co-authored-by: Uku Taht <Uku.taht@gmail.com>
5 years ago
Kian-Meng Ang 70cb3eed54
Fix typos (#1829) 5 years ago
Uku Taht 06b165eb6d Run GA import in monthly batches 5 years ago
Uku Taht 7c755d7c6d Forget has_stats 5 years ago
Uku Taht a359271af2 Fix more tests 5 years ago
Uku Taht 25baa6bae9 Fix tests 5 years ago
Uku Taht 333de87ceb Add stats_start_date field 5 years ago
Uku Taht 9308092184 Fix test 5 years ago
Uku Taht d40faf6ec7 More datepicker improvements 5 years ago
Uku Taht 7f58e6be4e Add Year to Date option in datepicker 5 years ago
Uku Taht 8467979051 Add All Time option to datepicker 5 years ago
Uku Taht ed871e7a0b Parse scientific notation in GA import 5 years ago
Uku Taht 8719865a41 Redirect user when they've already upgraded 5 years ago
Uku Taht ce649966ed Fix display 5 years ago
Uku Taht ebdd856df7 Fix tests 5 years ago
Andrea Mazzarella 3c93a2d91b
1421 contains filter (#1799)
* small refactors and adds 'contains' to modal

* supports contains filter in the backend

* moves entry and exit page under the page filter

* prettier

* updates the CHANGELOG

* undo package-lock changes

* fixes formatting for elixir

* renames unused parameter to _

* Update changelog

* Use uppercase for constants and update type/prefix lookup
5 years ago
RobertJoonas e5cf800d4a
Improve stats api (#1759)
* can now query events metric when breaking down on a visit prop

* updated changelog

* fixed formatting

* fixed a bug - querying for bounce rate does not return visits anymore

* removed the changelog update
5 years ago
RobertJoonas 492f47ba1e
Crm transfer data (#1749)
* pull from master

* added query generation by struct fields

* ready, improved tests

* fixed a naming mistake
5 years ago
Uku Taht 51cd24bcaf Allow user to cancel ongoing import 5 years ago
Uku Taht 203f87520b
Ga import improvements (#1784)
* Do not link Google account for import

* Record start date

* Fix tests
5 years ago
Uku Taht 59acff02f8 Fix tests 5 years ago
Uku Taht e1d6d119b0 Add end date to GA import 5 years ago
Uku Taht e026148fb0 Show better error message on duplicate domain
Co-authored-by: Marko Saric <me@markosaric.com>
5 years ago
Uku Taht e27734ed79
[Continued] Google Analytics import (#1753)
* Add has_imported_stats boolean to Site

* Add Google Analytics import panel to general settings

* Get GA profiles to display in import settings panel

* Add import_from_google method as entrypoint to import data

* Add imported_visitors table

* Remove conflicting code from migration

* Import visitors data into clickhouse database

* Pass another dataset to main graph for rendering in red

This adds another entry to the JSON data returned via the main graph API
called `imported_plot`, which is similar to `plot` in form but will be
completed with previously imported data.  Currently it simply returns
the values from `plot` / 2. The data is rendered in the main graph in
red without fill, and without an indicator for the present. Rationale:
imported data will not continue to grow so there is no projection
forward, only backwards.

* Hook imported GA data to dashboard timeseries plot

* Add settings option to forget imported data

* Import sources from google analytics

* Merge imported sources when queried

* Merge imported source data native data when querying sources

* Start converting metrics to atoms so they can be subqueried

This changes "visitors" and in some places "sources" to atoms. This does
not change the behaviour of the functions - the tests all pass unchanged
following this commit. This is necessary as joining subqueries requires
that the keys in `select` statements be atoms and not strings.

* Convery GA (direct) source to empty string

* Import utm campaign and utm medium from GA

* format

* Import all data types from GA into new tables

* Handle large amounts of more data more safely

* Fix some mistakes in tables

* Make GA requests in chunks of 5 queries

* Only display imported timeseries when there is no filter

* Correctly show last 30 minutes timeseries when 'realtime'

* Add with_imported key to Query struct

* Account for injected :is_not filter on sources from dashboard

* Also add tentative imported_utm_sources table

This needs a bit more work on the google import side, as GA do not
report sources and utm sources as distinct things.

* Return imported data to dashboard for rest of Sources panel

This extends the merge_imported function definition for sources to
utm_sources, utm_mediums and utm_campaigns too. This appears to be
working on the DB side but something is incomplete on the client side.

* Clear imported stats from all tables when requested

* Merge entry pages and exit pages from imported data into unfiltered dashboard view

This requires converting the `"visits"` and `"visit_duration"` metrics
to atoms so that they can be used in ecto subqueries.

* Display imported devices, browsers and OSs on dashboard

* Display imported country data on dashboard

* Add more metrics to entries/exits for modals

* make sure data is returned via API with correct keys

* Import regions and cities from GA

* Capitalize device upon import to match native data

* Leave query limits/offsets until after possibly joining with imported data

* Also import timeOnPage and pageviews for pages from GA

* imported_countries -> imported_locations

* Get timeOnPage and pageviews for pages from GA

These are needed for the pages modal, and for calculating exit rates for
exit pages.

* Add indicator to dashboard when imported data is being used

* Don't show imported data as separately line on main graph

* "bounce_rate" -> :bounce_rate, so it works in subqueries

* Drop imported browser and OS versions

These are not needed.

* Toggle displaying imported data by clicking indicator

* Parse referrers with RefInspector

- Use 'ga:fullReferrer' instead of 'ga:source'. This provides the actual
  referrer host + path, whereas 'ga:source' includes utm_mediums and
  other values when relevant.
- 'ga:fullReferror' does however include search engine names directly,
  so they are manually checked for as RefInspector won't pick up on
  these.

* Keep imported data indicator on dashboard and strikethrough when hidden

* Add unlink google button to import panel

* Rename some GA browsers and OSes to plausible versions

* Get main top pages and exit pages panels working correctly with imported data

* mix format

* Fetch time_on_pages for imported data when needed

* entry pages need to fetch bounces from GA

* "sample_percent" -> :sample_percent as only atoms can be used in subqueries

* Calculate bounce_rate for joined native and imported data for top pages modal

* Flip some query bindings around to be less misleading

* Fixup entry page modal visit durations

* mix format

* Fetch bounces and visit_duration for sources from GA

* add more source metrics used for data in modals

* Make sources modals display correct values

* imported_visitors: bounce_rate -> bounces, avg_visit_duration -> visit_duration

* Merge imported data into aggregate stats

* Reformat top graph side icons

* Ensure sample_percent is yielded from aggregate data

* filter event_props should be strings

* Hide imported data from frontend when using filter

* Fix existing tests

* fix tests

* Fix imported indicator appearing when filtering

* comma needed, lost when rebasing

* Import utm_terms and utm_content from GA

* Merge imported utm_term and utm_content

* Rename imported Countries data as Locations

* Set imported city schema field to int

* Remove utm_terms and utm_content when clearing imported

* Clean locations import from Google Analytics

- Country and region should be set to "" when GA provides "(not set)"
- City should be set to 0 for "unknown", as we cannot reliably import
  city data from GA.

* Display imported region and city in dashboard

* os -> operating_system in some parts of code

The inconsistency of using os in some places and operating_system in
others causes trouble with subqueries and joins for the native and
imported data, which would require additional logic to account for. The
simplest solution is the just use a consistent word for all uses. This
doesn't make any user-facing or database changes.

* to_atom -> to_existing_atom

* format

* "events" metric -> :events

* ignore imported data when "events" in metrics

* update "bounce_rate"

* atomise some more metrics from new city and region api

* atomise some more metrics for email handlers

* "conversion_rate" -> :conversion_rate during csv export

* Move imported data stats code to own module

* Move imported timeseries function to Stats.Imported

* Use Timex.parse to import dates from GA

* has_imported_stats -> imported_source

* "time_on_page" -> :time_on_page

* Convert imported GA data to UTC

* Clean up GA request code a bit

There was some weird logic here with two separate lists that really
ought to be together, so this merges those.

* Fail sooner if GA timezone can't be identified

* Link imported tables to site by id

* imported_utm_content -> imported_utm_contents

* Imported GA from all of time

* Reorganise GA data fetch logic

- Fetch data from the start of time (2005)
- Check whether no data was fetched, and if so, inform user and don't
  consider data to be imported.

* Clarify removal of "visits" data when it isn't in metrics

* Apply location filters from API

This makes it consistent with the sources etc which filter out 'Direct /
None' on the API side. These filters are used by both the native and
imported data handling code, which would otherwise both duplicate the
filters in their `where` clauses.

* Do not use changeset for setting site.imported_source

* Add all metrics to all dimensions

* Run GA import in the background

* Send email when GA import completes

* Add handler to insert imported data into tests and imported_browsers_factory

* Add remaining import data test factories

* Add imported location data to test

* Test main graph with imported data

* Add imported data to operating systems tests

* Add imported data to pages tests

* Add imported data to entry pages tests

* Add imported data to exit pages tests

* Add imported data to devices tests

* Add imported data to sources tests

* Add imported data to UTM tests

* Add new test module for the data import step

* Test import of sources GA data

* Test import of utm_mediums GA data

* Test import of utm_campaigns GA data

* Add tests for UTM terms

* Add tests for UTM contents

* Add test for importing pages and entry pages data from GA

* Add test for importing exit page data

* Fix module file name typo

* Add test for importing location data from GA

* Add test for importing devices data from GA

* Add test for importing browsers data from GA

* Add test for importing OS data from GA

* Paginate GA requests to download all data

* Bump clickhouse_ecto version

* Move RefInspector wrapper function into module

* Drop timezone transform on import

* Order imported by side_id then date

* More strings -> atoms

Also changes a conditional to be a bit nicer

* Remove parallelisation of data import

* Split sources and UTM sources from fetched GA data

GA has only a "source" dimension and no "UTM source" dimension. Instead
it returns these combined. The logic herein to tease these apart is:

1. "(direct)" -> it's a direct source
2. if the source is a domain -> it's a source
3. "google" -> it's from adwords; let's make this a UTM source "adwords"
4. else -> just a UTM source

* Keep prop names in queries as strings

* fix typo

* Fix import

* Insert data to clickhouse in batches

* Fix link when removing imported data

* Merge source tables

* Import hostname as well as pathname

* Record start and end time of imported data

* Track import progress

* Fix month interval with imported data

* Do not JOIN when imported date range has no overlap

* Fix time on page using exits

Co-authored-by: mcol <mcol@posteo.net>
5 years ago
RobertJoonas b4992cedc1
Referrer spam blocklist (#1750)
* integrating blocklist library

* loads blocklist dependency from Github
5 years ago
RobertJoonas 6b22806e1f
Allow admin access to locked dashboards (#1710)
* added super-admin access to locked dashboards

* fixed formatting
5 years ago
Uku Taht b0b9b9212b Fixes password authentication issue with shared links 5 years ago
Uku Taht 5b893bde39 Revert "Include 0 length pageviews in time_on_page calculation"
This reverts commit cee79e7875.
5 years ago
Uku Taht cee79e7875 Include 0 length pageviews in time_on_page calculation 5 years ago
Ralf Zimmermann 408d95fe09
Add custom goal props to breakdown endpoint (#1578) 5 years ago
Uku Taht 0505de9b42 Add site limit to enterprise upgrade plan 5 years ago
Uku Taht 52067803df Update session stats from event 5 years ago
ssendev 8077671f8a
Update session screen_size if unknown (#1610) 5 years ago
Uku Taht 0a5f6c5886 Implement wildcard filters for the API 5 years ago
Uku Taht cd40579740 Ignore unknown country code 5 years ago
Uku Taht 35ee2fe3fb Parse bunny header 5 years ago
Matt Colligan 8148d41e42
Add cities and regions to exported CSV data (#1485) 5 years ago
Uku Taht 60f116facd Fix tests 5 years ago
Uku Taht 0b253421ce Fix tests 5 years ago
Uku Taht 711cbdfca2 Remove unused custom domain code 5 years ago
Uku Taht 15931b9346 Add tests for utm_term and content in the API 5 years ago
Uku Taht 4ea0e29620 Add UTM content and term to CSV export 5 years ago
Uku Taht 48ad7485c8
PR 1393 continued (#1542)
* Add `utm_content` and `utm_term`.
Support `utm_content` and `utm_term` as requested in #515.

* Add dropdown for UTM options

* Remove utm_content and term from filter modal for now

Co-authored-by: Blender Defender <defenderblender@gmail.com>
5 years ago
Uku Taht dc8d20496c Show locked message in shared link 5 years ago
Ralf Zimmermann faf4459b15
Add "events" metrics to public breakdown endpoint (#1529) 5 years ago
Uku Taht 66575f497b Add goal type to JSON response 5 years ago
Uku Taht 97cdc4814e Refactor goals API 5 years ago
Uku Taht 1dba113e2f
[Draft] Improve location translations (#1526)
* WIP

* Use location library for search suggestions

* Remove unused code

* Remove Countries completely

* Fix tests
5 years ago
Uku Taht b49fd19934
Pull 1506 - formatted (#1527)
* adding api route PUT /api/v1/sites/goals with form fields "goal_type" and "goal_value" with supported types "event" and "page"

* arm64 docker images

* adding api route DELETE /api/v1/sites/goals/:goal_id with form param "site_id"

* revert makfile + package.json

* return statement hotfix in case site could not be found

* adding api route PUT /api/v1/sites/goals/:goal_id with form params "site_id", "goal_type", and "goal_value"

* update the goal api routes to accept event_name or page_path instead of goal_value

* cleaning goals model

* mix format

Co-authored-by: Ahmed Abbas <a.abbas@ixdc.net>
5 years ago
RobertJoonas 9022234aa6
added invoices section (#1511)
* added invoices section

* fixed formatting

* UI and code style improvements, paddle sandbox setup only on dev environment (fixed)
5 years ago
Uku Taht a2d0244315 Fix site locked email 5 years ago
Uku Taht 01706b7590 Remove dead code 5 years ago
Uku Taht 4d0bc61ffd Remove Twitter stuff 5 years ago
Uku Taht e7e36b994d Skip usage checking for users who are on grace period 5 years ago
Uku Taht effe56b3e4 Use new iso_codes package 5 years ago
Uku Taht 7cfd9f95f1 Delete a site through the API 5 years ago
Uku Taht 07af25ffc3 Fix subscription_interval for enterprise plans 5 years ago
Uku Taht 9c07cf7ea6 Report allowance issues to Sentry 5 years ago
Uku Taht 2463296403 Fetch enterprise plan corretly 5 years ago
Uku Taht 5880678eaf Fix tests 5 years ago
Uku Taht 7e93500834 Allow admins to access the stats API 5 years ago
Uku Taht 2bdfec1cc0
JS refactor: use generic ListReport for country report (#1487)
* Use ListReport for countries

* Fix countries tests

* Replace Browsers with ListReport

* Use Listreport for OS and screen size
5 years ago
Uku Taht 05bf43c1be
City level location data (#1449)
* Merge branch 'plausible_master'

* Add City level details

* Add City level details

* Use ISO codes instead of geoname_id for subdivisions

* Add easier way to configure geolocation database

* Add workflow for dev branch

* Correct clickhouse migration

* Translate subdivision names

* Translate city names

* WIP

* Region and country filters

* Fix region filter

* Remove region_name when removing region filter

* Add modals for regions and cities

* Remove dead code

* WIP

* Revert "WIP"

This reverts commit 3202bf2fe9.

* Feature flag to hide cities when deployed

* Add changelog entry

* Remove unused code

* Remove unused variables

* Fix test

Co-authored-by: AymanTerra <aymanterra@yahoo.com>
5 years ago
Uku Taht 06d0d0eafa Fix over limit email reccommendation 5 years ago
Uku Taht e355828465 Remove pageview filter by default 5 years ago
Uku Taht 6b55ace014 Fix error with JSON parsing 5 years ago
Uku Taht 0edf4765d4 Ignore maps in prop values 5 years ago
Uku Taht d3311898c5 Show message when grace period is over 5 years ago
Uku Taht e9cb8eb4e2 Remove grace period if user upgrades 5 years ago
Uku Taht ec2a4a1a65 Add [Action required] to over limit email 5 years ago
Uku Taht d547d0f996 Send email notification when sites are locked at grace period end 5 years ago
Uku Taht 29cb7462e6 Add grace period to upgrade 5 years ago
Matt Colligan 7faa2f6673
CSV export output conversions and conversion rate when filtering for goal (#1464)
* Only add percentages to dashboard data when not filtering goal

* Correctly name CSV headers when exporting conversion data

* Remove percentages from tests when filtering for goal
5 years ago
Matt Colligan 3380685d40
Fix custom property total conversions value not displayed & Export custom properties (#1456)
* Fix custom property total conversions value not displayed

The custom property conversion metrics are not consistent with the other
metrics resulting in the total conversions not being displayed in the
dashboard. This fixes that.

* Export custom props of current goal when filtering dashboard for goal

This makes the CSV export also output a `prop_breakdown.csv` file which,
for the currently filtered goal, contains the conversion data for each
of its configured properties.

* Add test for goal-filtered CSV export
5 years ago
Uku Taht 7a02aae2a5 Accept custom props in expected format 5 years ago
Matt Colligan d781301886 Apply existing filter to visitors.csv in export (#1454)
Also adds a test to ensure the filter always applies.
5 years ago
Matt Colligan 29e37c8342 More improvements for CSV export (#1446)
* Round time_on_page metric returned by 'pages' API

This rounds the `time_on_page` metric returned as part of the `pages`
API to the nearest second. While this shows no apparent change in the
web UI, it removes the decimal from the exported data in `pages.csv`.

* Tidy up export tests

* Round time_on_page in db query

* Add goal to CSV export tests

* Display CSV export spinner until download is ready

The mechanism used to make the page aware that the download is ready is:

- Client code sets a cookie and requests download.
- Server code handles download and removes cookie when complete.
- Client code polls every 1s to check the cookies, removing spinner when
  the export cookie is removed.
5 years ago
Matt Colligan 945bf82c72
Fix conversions metrics (#1447)
f576fa2 should have updated the conversion metric names so that
`unique_conversions` and `total_conversions` are the two metrics
returned by the conversions API. This updates those so that the CSV
export outputs the correct data.
5 years ago
Matt Colligan f576fa2a2c
Improvements to CSV export (#1427)
* Add details=True to export API parameters

This makes the ZIP export add `%{"details" => "True"}` to the query's
`params` when fetching data internally for packaging in the ZIP.

This adds bounce_rate and time_on_page to the data in pages.csv, and
bounce_rate and visit_duration to sources.csv.

* Make API return data with consistent names

Some of the data types returned via the JSON or CSV API use inconsistent
naming, and some have redundant name changes (i.e. count -> visitors ->
count). This makes these all consistent and removes the redundancy.

This addresses #1426, fixes some of the CSV headers, and unifies the
JSON and CSV return data labels.

* Update changelog

* Test should use Timex.shift, not relative time

* Return full country names in CSV export

This also replaces the " character with ' in two country names, as those
are the characters used in the names, yielding a more predictable and
'correct' output.

* Fetch CSV exported data concurrently

* Use spinner to indicate when export has started

* Use 300 as default number of brekadown entries for export

Higher numbers (e.g. 1000) seem to cause clickhouse errors when there
many pages to request. It is unclear what is causing the error, as
clickhouse returns an "unknown" error code and an empty error message.
5 years ago
Matt Colligan 8138a6e3a5
Start putting together means to export all aggregate data (#1387)
* Export all dashboard data in zip

This packages all data currently visible in the dashboard into
individual CSVs and downloads them together in a zip.

* Also export conversions in zip of CSVs

* Update export test with zip file response

* Add zip file download to changelog
5 years ago
Uku Taht 7622621a1d Adds test 5 years ago
Uku Taht 1b0323f5c1 Use more path helpers 5 years ago
Uku Taht 710f6f568a Ignores invalid query param part 5 years ago
Uku Taht 6dd9b0bb2f Do not error when bad custom prop value is supplied 5 years ago
Uku Taht b9b37cdd34 Show pricing v2 when appropriate 5 years ago
Uku Taht 26e34596c8 Allow enterprise plan users to create sites 5 years ago
Uku Taht e66e9bd1b7 Check site limit for enterprise customers 5 years ago
Uku Taht 792e534edd Optimize usage calculation for large accounts 5 years ago
Uku Taht b1594c631e Adjust API key limits for enterprise plans 5 years ago
Uku Taht 6a5b383e2b Add enterprise plans 5 years ago
Vignesh Joglekar 7e9d83d62e
Adds pagination for custom props & fixes an issue with errant (none) return values (#1382)
* Adds pagination for custom props, fixes errant (none) being returned in the middle of a dataset

* Formatting

* Fixes errant (none) value correctly

* Changelog

* Adds tests
5 years ago
Uku Taht 2e6f13ba67 Make sure ownership transfer sets trial_expiry_date 5 years ago
Uku Taht cd68761928 Fix utm_sources when filtering by event:page 5 years ago
Uku Taht bd7de59a9c Show total visitors when filtered for goal 5 years ago
Uku Taht a722d02e63 Return 400 when domain is missing 5 years ago
Uku Taht 5be38a57fa Do not return prop names when they are not used 5 years ago
Uku Taht f42f4da44b Fix how sampling is applied when filtering for a goal 5 years ago
Uku Taht 2f3bee800d Return more information in /api/event response
Fixes #1246
5 years ago
Uku Taht d93fe8197e Accept chrome extension URLs 5 years ago
Uku Taht c913d2fbb9 Fix referrer drilldown for Google
Fixes #1341
5 years ago
Ro Savage b3bc796d50
Add conversion_rate to sources api and source table (#1299)
* Add conversion_rate to sources api and source table

* Remove percentageFormatter

* Update source tests to include conversionat rate

* Add CR to detals modal

* Correct formatting with linter

* Add change log

* Add CR to Pages, Device and Countries panels
5 years ago
RobertJoonas 4742e05ab1
Allow URLs without hostname setting it to (none) (#1335) 5 years ago
Uku Taht 7e772151e3 Adjust tests 5 years ago
Uku Taht bd797c201f Mix format 5 years ago
Uku Taht 7cb148a4f1 Use only root domain in user id generation 5 years ago
Uku Taht c68b88c24b
Upgrade otp (#1313)
* update tool versions to latest

* Upgrades erlang and elixir, format project

Co-authored-by: happysalada <raphael@megzari.com>
5 years ago
Uku Taht 700a65c98a
Remove trial banner for admins & viewers (#1308)
* Start trial only when the user creates a site

* End trial when ownership is transfered
5 years ago
Uku Taht c8a1b5c73c Do not cascade user deletion to sites they do now own 5 years ago
Uku Taht 33ba5d79b6
Memoize has_stats? (#1302)
* Memoize has_stats?

* Remove unused alias
5 years ago
Uku Taht 01412d090f Fix sampling in JOIN queries 5 years ago
Uku Taht 77d3d8695d Remove glob filters from API 5 years ago
Uku Taht 0fe053e106 Fix realtime dashboard 5 years ago
Uku Taht 95edbc0ab5 Fix test 5 years ago
Uku Taht a9b16fa6c6 Fix exit rate when filtering for page 5 years ago
Uku Taht 3d3601c332 Support negated glob filters everywhere 5 years ago
Uku Taht 49214eaff8 Fix exit pages when using a goal filter 5 years ago
Uku Taht 7e78157c4b
Merge branch 'master' into stats-module 5 years ago
Uku Taht 7d1b1214e2 Implement dashboard with new stats module 5 years ago
Uku Taht a5c7e4f7ef mix format 5 years ago
Uku Taht 13d0dce901 Make invitations email case insensitive 5 years ago
Uku Taht 0de89bad82
Add filter groups (#1167)
* Add filter groups

* New flow for filters

* Visual consistency

* Mobile improvements to dropdown

* Do not let filters wrap on a new line

* Fix country filter

* Add mix format to pre-commit configuration

* Make eslint happy

* Fix formatting

* Use transition from headlessUI
5 years ago
Uku Taht fef5337a64 Make sure only the current subcription is used in expiry notifications 5 years ago
Uku Taht 70c6010538 Check lock status after subscription has been updated 5 years ago
Vignesh Joglekar 30ac901141
Adds manual, editable, auto-suggested filters, and negated&globbed path-based filters (#1121)
* Adds manual-filters + friends

commit 308192044d726e9a6f7406e333048ab3407aa260
Merge: 39287ab a299fab
Author: Vignesh Joglekar <rexvigasaurus@gmail.com>
Date:   Fri Jun 11 03:40:55 2021 -0500

    Merge branch 'manual-filters' into manual-filters-2

commit a299fab1fb
Author: Vignesh Joglekar <rexvigasaurus@gmail.com>
Date:   Fri Jun 11 02:20:26 2021 -0500

    Changes to split and pattern matched function for time_on_page

commit 10f10c933a
Author: Vignesh Joglekar <rexvigasaurus@gmail.com>
Date:   Fri Jun 11 01:53:18 2021 -0500

    Fixes a couple of minor UX issues

commit f2e5ce8eb3
Author: Vignesh Joglekar <rexvigasaurus@gmail.com>
Date:   Fri Jun 11 01:49:10 2021 -0500

    Fixes time on page for globbed and negated page paths

commit bb18af6526
Author: Vignesh Joglekar <rexvigasaurus@gmail.com>
Date:   Thu Jun 10 05:24:05 2021 -0500

    Close to finalized version of updated version

    Just needs some additional testing + potentially code cleanup

commit d0b7bfe3da
Author: Vignesh Joglekar <rexvigasaurus@gmail.com>
Date:   Fri May 28 04:21:21 2021 -0500

    Real Dialyzer Fix

commit 296a76af34
Author: Vignesh Joglekar <rexvigasaurus@gmail.com>
Date:   Fri May 28 03:44:29 2021 -0500

    Dialyzer fix

commit 91f3b44017
Author: Vignesh Joglekar <rexvigasaurus@gmail.com>
Date:   Thu May 27 17:11:24 2021 -0500

    Changelog

commit e041f75745
Author: Vignesh Joglekar <rexvigasaurus@gmail.com>
Date:   Thu May 27 17:09:52 2021 -0500

    Formatting

commit f689642204
Merge: e00929b 4ff25f6
Author: Vignesh Joglekar <rexvigasaurus@gmail.com>
Date:   Thu May 27 17:08:47 2021 -0500

    Merge branch 'master' into manual-filters

commit e00929b1c5
Merge: 83887c4 806975e
Author: Vignesh Joglekar <rexvigasaurus@gmail.com>
Date:   Thu May 27 17:01:48 2021 -0500

    Merge branch 'manual-filters' of github.com:Vigasaurus/plausible-analytics into manual-filters

commit 83887c49cc
Author: Vignesh Joglekar <rexvigasaurus@gmail.com>
Date:   Thu May 27 16:59:32 2021 -0500

    Adds tests for suggestions, formats goals suggestion query

commit 1cb7732a08
Author: Vignesh Joglekar <rexvigasaurus@gmail.com>
Date:   Thu May 27 16:36:31 2021 -0500

    Adds goals as auto-complete capable filter

commit 4ca39cc406
Author: Vignesh Joglekar <rexvigasaurus@gmail.com>
Date:   Thu May 27 16:08:22 2021 -0500

    95% completed auto-complete setup

    Still needs:
    - tests
    - goals added as filter

commit 22d38c4cd8
Author: Vignesh Joglekar <rexvigasaurus@gmail.com>
Date:   Thu May 27 03:47:19 2021 -0500

    80% of auto-complete filters progress

    Still needs
    - countries and screen into new format
    - re-style dropdown and background
    - drop debounce time
    - tests

commit 806975ede9
Merge: 81c5e05 1a93542
Author: Vignesh Joglekar <rexvigasaurus@gmail.com>
Date:   Tue May 25 15:28:21 2021 -0500

    Merge branch 'master' into manual-filters

commit 81c5e05760
Author: Vignesh Joglekar <rexvigasaurus@gmail.com>
Date:   Tue May 25 15:21:03 2021 -0500

    Makes colorings on top bar elements consistent

commit fa7f6c2268
Author: Vignesh Joglekar <rexvigasaurus@gmail.com>
Date:   Tue May 25 14:58:25 2021 -0500

    Makes requested changes, adds different version of filter button

commit 7dc65b9b7a
Author: Vignesh Joglekar <rexvigasaurus@gmail.com>
Date:   Sat May 22 04:29:01 2021 -0500

    Changelog

commit c684f1c76a
Author: Vignesh Joglekar <rexvigasaurus@gmail.com>
Date:   Sat May 22 04:26:14 2021 -0500

    Various UI Improvements

    - Makes edit buttons full-length & properly sized
    - Adds remove filter button in edit menu

commit a632e7a8ee
Author: Vignesh Joglekar <rexvigasaurus@gmail.com>
Date:   Sat May 22 03:11:50 2021 -0500

    Adds tests for exclusions and wildcards

commit eb91a7942b
Author: Vignesh Joglekar <rexvigasaurus@gmail.com>
Date:   Sat May 22 03:02:23 2021 -0500

    Fixes editing UX on list view

commit 6209d72aee
Author: Vignesh Joglekar <rexvigasaurus@gmail.com>
Date:   Fri May 21 04:01:17 2021 -0500

    Bugfix in realtime view, formatting

commit 007d44df38
Author: Vignesh Joglekar <rexvigasaurus@gmail.com>
Date:   Fri May 21 03:23:16 2021 -0500

    Second pass - mostly everything user-facing is done

    Still needs:
    - Tests
    - Potentially negating other filters
    - Potentially some code cleanup

commit cb7b5b9fbd
Author: Vignesh Joglekar <rexvigasaurus@gmail.com>
Date:   Fri May 21 01:49:52 2021 -0500

    First pass on manual filter & path regex/negated filters

    Still needs:
    - Form structure on filter modal
    - Edit filter button
    - Filter dropdown UI improvement
    - Filter modal mount data collection
    - Tests
    - Potentially negating other filters

* Fixes an issue between the updated React version and Flatpickr not showing up

Reverting to v16 was the simplest fix without diving into the issue upstream with Flatpickr

* Resolves comments on UI and code style

Co-authored-by: Uku Taht <Uku.taht@gmail.com>
5 years ago
Uku Taht 883841f789 Make sure the latest subscription is always used 5 years ago
Uku Taht 27f99da8ad Do not send email reports for locked sites 5 years ago
Uku Taht e71de6dc1f
Invitations (#1122)
* Invite existing user to a site

* Add invitation flow for non-existing users

* Accept and reject invitations

* Use invitation flow for existing users

* Locking mechanism for sites

* Authorization for site settings

* Show usage based on site ownership

* Add ability to remove members from a site

* Do not show settings link to viewer roles

* Ability to remove invitations

* Remove `Plausible.Sites.count_for/1`

* Fix tests

* Do not show the trial banner after the trial

* Correct trial emails

* Transfer ownership

* Send invitation email to existing user

* Add invitation email flows

* Add plug for role-based authorization

* Rename AuthorizeStatsPlug -> AuthorizeSiteAccess

* Add email flow for ownership transfer

* Fix URLs in emails

* Fix small copy issues

* Make 'People' its own section in site settings

* Notify user via email if their access has been removed

* Check site lock status when invitation is accepted

* Check lock status when user subscribes

* Make sure only admins and owners can create shared links

* Changelog

* Add LockSites to daily cron

* Clean invitations after 48 hours

* Add notices about expiry

* Add invitation expired page

* Add doc link
6 years ago
Uku Taht c0083815d4 Fix IP address parsing for ipv6 with port number 6 years ago
Uku Taht fa60ebd6b0 Scrub port from x-forwarded-for 6 years ago
Uku Taht ad6faece87
Stats API: Support IN filter expression (#1090)
* Support IN filter expression

* Update clickhouse_ecto
6 years ago
Uku Taht 9f16603bfb Accept application/json in /api/event 6 years ago
Uku Taht c37a2de627 Revert "Adds manual filters on FE, capability for exclusion filters (and the globbing we're used to) for path-based filters (#1067)"
This reverts commit b6eeb40472.
6 years ago
Vignesh Joglekar b6eeb40472
Adds manual filters on FE, capability for exclusion filters (and the globbing we're used to) for path-based filters (#1067)
* First pass on manual filter & path regex/negated filters

Still needs:
- Form structure on filter modal
- Edit filter button
- Filter dropdown UI improvement
- Filter modal mount data collection
- Tests
- Potentially negating other filters

* Second pass - mostly everything user-facing is done

Still needs:
- Tests
- Potentially negating other filters
- Potentially some code cleanup

* Bugfix in realtime view, formatting

* Fixes editing UX on list view

* Adds tests for exclusions and wildcards

* Various UI Improvements

- Makes edit buttons full-length & properly sized
- Adds remove filter button in edit menu

* Changelog

* Makes requested changes, adds different version of filter button

* Makes colorings on top bar elements consistent
6 years ago
Uku Taht 1a93542cd7 Add rate limit to API requests 6 years ago
Uku Taht 6228995d4f Rate limit login attempts to 5 per minute 6 years ago
Uku Taht 120c4060a8 Remove empty test file 6 years ago
Uku Taht 2cf0c1a456 Use `forwarded` header when X-Forwarded-For is missing 6 years ago
Vignesh Joglekar 41e4690116
Adds Time on Page metric to Top Pages report (#1007)
* First pass

Needs more testing & potentially cleanup

* Fixes tests, error handling

* Formatting

* Removes broken test

* Fixes inconsistent test

This was due to Clickhouse setup not inserting the sessions with the exact same timestamp consistently and making the test inconsistent

* Combines `include` param, asyncs time_on_page and bounce_rate

* Fixes CH error when no pageviews exist in period

* Format

* Changelog

* Increases await timeout to accomodate larger data sets

* Improves handling of timeout behavior

* Fixes session-based filtering on time on page queries

* Formatting

* Removes old forced entry page modal from top pages report
6 years ago
Uku Taht f16666e374 Localize billing screens 6 years ago
Uku Taht 793957ed90 fix check usage test 6 years ago
Uku Taht d33fe1cf97 Use new pricing in trial email test 6 years ago
Uku Taht 7c520e16d2 Make discount notices more generic 6 years ago
Uku Taht 1b91909021 Grandfather old plans 6 years ago
Uku Taht 885b1d271a Refactor current plan data format to make changing it easier 6 years ago
Uku Taht e511c9663e
Pagination for sites listing (#994)
* Add simple pagination to sites index page

* Add changelog entry
6 years ago
Uku Taht 6492b61fa5 Add test for grandfathering 6 years ago
Uku Taht 7f9057818d Protect the API endpoint from creating too many sites 6 years ago
Uku Taht 119b9514b2 Add limit of 20 sites 6 years ago
Uku Taht 5efa15f381 Use CF-Connecting-IP when present for remote IP address 6 years ago
Uku Taht 0799138622 Add nil check to hashing 6 years ago
Uku Taht d30f6def0f Add presence check when unsubscribing from email reports 6 years ago
Uku Taht 07653079e8 Don't fail oban job when provision_ssl_certificates fails 6 years ago
Uku Taht 5c77ec3f5d
Support multiple `data-domain` values (#968)
* Support multiple `data-domain` values

* Add changelog entry
6 years ago
Uku Taht 8d1039b8ce
Upgrades Oban to v2.6.1 (#967)
* Upgrades Oban to v2.6.1

* Use correct clickhouse_ecto
6 years ago
Uku Taht b107717774
Additional metrics for timeseries API call and CSV export (#952)
* Add `metrics` option to timeseries call

* Add more metrics to csv export

* Add changelog entry
6 years ago
Uku Taht b8baefee3b
Fix weekly report time range (#951)
* Fix weekly report time range

* Update changelog
6 years ago
Uku Taht 6ca7425dc9
Send renewal notification for annual subscriptions (#949)
* Add background job to send renewal notification

* Use copy from Marko

* Add expiration email in case the user has cancelled

* Add queue config

* Use correct tag for expiration email
6 years ago
Uku Taht 86d73b68b8 Fix formatting 6 years ago
Uku Taht edffe64a39 Merge branch 'provision-api' 6 years ago
Uku Taht 70f0657327 Update shared link API 6 years ago
Uku Taht 956e92f161 Change default timezone to Etc/UTC 6 years ago
Uku Taht 70cc25e8cd Increase ssh timeout in certbot call 6 years ago
Uku Taht 9de0902038 Fix test 6 years ago
Uku Taht 80abfa6ef6 Add validations 6 years ago
Uku Taht d473accf40 Add API key scopes 6 years ago
Uku Taht 3ed964b45b Add API endpoints for site and shared link creation 6 years ago
Uku Taht 48be0e3fda Revert "Add UTM tags to login links from email reports"
This reverts commit 091fc423f9.
6 years ago
Uku Taht 091fc423f9 Add UTM tags to login links from email reports 6 years ago
Uku Taht 58cff47b6f
Add name to shared links (#910)
* Add name to shared links

* Add changelog entry
6 years ago
Uku Taht e12b9fa42c Use GET instead of POST for logout
Fixes #730
6 years ago
Uku Taht 1d4ae3d16f Fix tests 6 years ago
Uku Taht e315d0cf38
Fix shared link download link (#884)
* Fix shared link download link

* Add test and changelog entry

* Format
6 years ago
Uku Taht c4c0bb2095 Various formatting improvements
* Format numbers over 1B
* Show full number when hovering
6 years ago
Uku Taht fb64345e92 Format numbers larger than 100m 6 years ago
Uku Taht 3eb2f6d7a8
Add support for sampling (#875)
* Add support for sampling

* Use correct mix package

* Add notice about data sampling
6 years ago
Uku Taht 432ad2a606
Stats API improvements (#826)
* Show hourly data for a day

* Fix aggregations with long periods and comaprisons enabled

* Return Direct / None instead of empty string when missing source

* Fix/Align breakdown query with current queries in the UI

* Show correct number of days for 7d and 30d

* Wrap results of aggregate call in `results`

* Allow breakdown by event:name

* Ensure event:name filter works in all contexts

* WIP

* Add support for custom event props
6 years ago
Victor Truong c29bcc89b5
Fix typo with "succesfully" (#836) 6 years ago
Uku Taht 844af698ce
Embeddable dashboard (#812)
* Add embedded mode for shared links

* Add test for X-Frame-Options

* Improve embed code generator

* Add changelog entry

Co-authored-by: aymanterra <aymanterra@users.noreply.github.com>
Co-authored-by: Brad Kane <bradkane@users.noreply.github.com>
6 years ago
Uku Taht 0ceb539a22 Reduce errors sent to Sentry 6 years ago
Vignesh Joglekar b952053cee
Adds same glob patterns as exclusions for pageview goals (#750)
* First Pass - 90% wrong

* Second pass - basically done

* Swaps N+1 for multiMatchAllIndices & friends

* New version of query with just one regex computation

* Slight formatting, basic test

* Updates tests to match master new events

* Changelog

* Adds more sophisticated glob tests
6 years ago
Uku Taht cda031d453
Bookmarkable shared links (#752)
* Implement shared links with a static URL

* Separate sessio cookie from shared link cookie
6 years ago
Uku Taht 9db24f54d5 Enable over limit notifications 6 years ago
Vignesh Joglekar ff32218bd0
Adds entry and exit pages to Top Pages module (#712)
* Initial Pass

* Adds support for page visits counting by referrer

* Includes goal selection in entry and exit computation

* Adds goal-based entry and exit page stats, formatting, code cleanup

* Changelog

* Format

* Exit rate, visit duration, updated tests

* I keep forgetting to format :/

* Tests, last time

* Fixes double counting, exit rate >100%, relevant tests

* Fixes exit pages on filter and goal states

* Adds entry and exit filters, fixes various bugs

* Fixes discussed issues

* Format

* Fixes impossible case in tests

Originally, there were only 2 pageviews for `test-site.com`,`/` on `2019-01-01`, but that doesn't make sense when there were 3 sessions that exited on the same site/date.

* Format

* Removes boolean function parameter in favor of separate function

* Adds support for queries that use `page` filter as `entry-page`

* Format

* Makes  loader/title interaction in sources report consistent
6 years ago
Uku Taht 8a455dd191
Prevent user id collisions in session tracking (#744) 6 years ago
Uku Taht 3c4931598b Fix breakdown call 6 years ago
Uku Taht a29ac44434
Api improvements (#723)
* Require date parameter with custom period

* Validate format of the `date` param with custom period

* Return proper status codes for auth failures

* Add breakdown endpoint

* Fix pagination

* Add API validations for breakdown call

* Change the breakdown endpoint to return an object instead of an array

* Remove change to aggregate call

* Change timeseries call
6 years ago
Uku Taht 9eff6b03ed Send test billing upgrade notifications 6 years ago
Uku Taht 76d8dc588c Get paddle last payment from api call 6 years ago
Uku Taht a8cb187e8d
Track billing cycles (#697)
* Add dummy check usage worker

* Add and persist last_bill_date

* Remove name clash

* Correct timing and suggestions for over limit emails

* Fix tests for trial upgrade notifications
6 years ago
Uku Taht 77628d04c9
Validate API query params (#703)
* Add validations for `date`, `period` and `metrics` variables

* Validate interval query parameter
6 years ago
Uku Taht 5acb5b7039
Stats API (#679)
* WIP

* Add ability to filter by anything

* Add API keys

* Add version to api endpoint

* Fix API test route

* Fix API tests

* Allow 'date' parameter in '6mo' and '12mo'

* Rename session -> visit in API filters

* Filter expressions in the API

* Implement filters in aggregate call

* Add `compare` option to aggregate call

* Add way to manage API keys through the UI

* Authenticate with API key

* Use API key in tests
6 years ago
Uku Taht a4bf74a84c Do not allow duplicate spike notification to be created 6 years ago
Uku Taht e7a7deef84
Clarify that custom events are billed separately from events (#607)
* Clarify that custom events are billed separately from events

* Fix for dark mode
6 years ago
Uku Taht b7a2b3b423 Cascade deletion of site to spike notifications 6 years ago
Uku Taht 563c8d99f4 Fix new configuration 6 years ago
Uku Taht e786af648d
Fix session duration for out-of-order events (#593) 6 years ago
Oliver Kriska 4d4f8ba5c3
Elixir & Docker upgrade (#566)
* Elixir upgrade

 - upgraded Elixir to 1.11.2
 - upgraded Erlang to 23.2.1
 - upgraded Phoenix to 1.5.7
 - Upgraded:
  bamboo_postmark 0.6.0 => 0.7.0 (minor)
  bcrypt_elixir 2.2.0 => 2.3.0
  csv 2.3.1 => 2.4.1
  ecto 3.4.6 => 3.5.5
  ecto_sql 3.4.4 => 3.5.3
  elixir_make 0.6.0 => 0.6.2
  ex_machina 2.4.0 => 2.5.0
  excoveralls 0.12.3 => 0.13.4 (minor)
  file_system 0.2.8 => 0.2.10
  gettext 0.18.0 => 0.18.2
  httpoison 1.6.2 => 1.7.0
  nanoid 2.0.2 => 2.0.5
  phoenix_ecto 4.1.0 => 4.2.1
  phoenix_live_reload 1.2.2 => 1.3.0
  postgrex 0.15.5 => 0.15.7
  sentry 7.2.4 => 7.2.5
  timex 3.6.2 => 3.6.3
  tzdata 1.0.3 => 1.0.5
  yamerl 0.8.0 => 0.8.1

* Elixir & Docker upgrade

upgraded to 1.11.3

* Docker

 - fixed missing python3 package for newer Alpine linux
6 years ago
Oliver Kriska ae42b86792
Dialyzer and Credo checks (#558)
* Checks

 - added Dialyzer
 - fixed Dialyzer errors
 - added Dialyzer check to GitHub Actions with cache
 - added Credo
 - fixed Credo Warnings
 - added Credo Warnings check to GitHub Actions with cache
 - added compile warnings check to GitHub Actions
 - reformated GitHub Actions YAML

* Dialyzer

 - allow it in test env

* Dialyzer

 - fixed test env
 - renamed GitHub actions steps

* AppSignal

 - upgraded deprecated version
 - Upgraded:
  appsignal 2.0.5 => 2.0.7
  certifi 2.5.2 => 2.5.3
  hackney 1.16.0 => 1.17.0
  idna 6.0.1 => 6.1.1
  parse_trans 3.3.0 => 3.3.1
  unicode_util_compat 0.5.0 => 0.7.0 (minor)

* Credo

 - fixed CRM plug
6 years ago
Uku Taht 9bbaa756b8 Fix user deletion 6 years ago
Uku Taht 81c12884cd
Add elixir action (#526)
* Add elixir action

* Format the codebase

* Add postgresql

* Postgres config

* Run postgres on localhost

* Add clickhouse to CI
6 years ago
Uku Taht ef4af9249f Fix double quote handling in Clickhouse 6 years ago
Uku Taht 49c4ec2e9c Encode site domain when redirecting a shared link
Fixes #376
6 years ago
Uku Taht 9c4493de66 Send welcome email after first site creation 6 years ago
Uku Taht aa7ae87811
Onboarding UX improvements (#441)
* WIP

* Actually activate the user

* Send email verification codes

* Send activation code with email

* Only show onboarding steps during first site creation

* Add worker to config

* Consistent form styles

* Send welcome email when user activates account

* Add changelog entry

* Use https in new site form

* Correct spelling in email
6 years ago
Uku Taht acce144809 Show noref in notification email 6 years ago
Uku Taht 35e4051016 Add more information to spike email 6 years ago
Uku Taht f35ef2e0b1 Add settings UI for spike notifications 6 years ago
Uku Taht 11c9bdc328 Make sure a notification is not sent again within 12 hours 6 years ago
Uku Taht 54f70717c4 Add basic spike notifications
Co-authored-by: Jerod Santo <jerod@changelog.com>
6 years ago
Uku Taht a92bca2c86 Fix sites list when user has no sites 6 years ago
Uku Taht df262266fb Fix settings tests 6 years ago
Uku Taht d71e40f16f Validate domain format on site creation 6 years ago
Uku Taht 19a9e314fc Accept anything in prop value as long as it can `to_string` 6 years ago
Uku Taht 1d4bddd79d Decode URLs properly 6 years ago
Uku Taht b0b5d92f6e Fix URL decoding 6 years ago
Uku Taht d0d7b823f8
Browser and OS version (#397)
* Collect browser and OS version

* Display browser version

* Show operating system versions

* Device categorization

* Treat headless chrome like a bot

* Ignore events from automated browsers

* Only take major and minor of the version

* Add tests

* Add CHANGELOG entry

* Add changelog entry for bots

* Store empty value as browser when unknown
6 years ago
Uku Taht 0a7684f3bc Mix format 6 years ago
Uku Taht b816d92020 Show conversion rates in conversions report 6 years ago
Uku Taht 044da18fec Fix tests for metadata 6 years ago
Uku Taht 40900c7653
Ability to add event metadata (#381)
* Ability to add event metadata

* Close Dropdown on outside click

* Show (none) value in metadata breakdown

* Allow filtering for metadata key/val pairs

* Use correct clickhouse_ecto

* Better naming for meta filter

* Add tests

* Add changelog entry

* Remove change made for testing
6 years ago
Uku Taht dd44bc803d
Do not error when activating an account that already exists (#370)
* Do not error when activating an account that already exists

* Add changelog entry
6 years ago
Uku Taht e228fd1f85
Additional filters (#358)
* Filter by screen size

* Filter by browsers

* Add filter for Operating system

* Add changelog entry

* Add filter for countries

* Add tests for filters
6 years ago
Preslav Rachev 431c02ad58
#332: Refactor stats controller (#337)
* #332: Refactor stats controller

* Simplify the conditional logic
6 years ago
Uku Taht 8d9667a949
Show utm_medium, utm_source, and utm_campaign in sources modal (#321)
* Show utm_medium, utm_source, and utm_campaign in sources modal

* Allow filtering by UTM tags

* Integrate filters with URL bar

* Add CHANGELOG entry

* Refresh modal when filter changes

* Remove Direct / None from campaign results

* Add UTM tabs to top sources report

* Add pagination

* Remove dropdown from top sources popup

* Fix bug in clickhouse_ecto

* Remove referrers_for_goal

* Make sure UTM tags work OK with goals

* Make source tab selection sticky

* Consistent styling for devices and source tabs

* Add noref in realtime to utm tabs

* Fix tests
6 years ago
Uku Taht 9ad2075d6f Send confirmation email when user cancels their subscription 6 years ago
Uku Taht 122dd94327
Store `utm_medium`, `utm_source`, and `utm_campaign` (#318) 6 years ago
Uku Taht 4b36bb7138
Use clickhouse_ecto for db connection (#317)
* Use clickhouse-ecto for stats

* Use clickhouse ecto instead of low-level clickhousex

* Remove defaults from event schema

* Remove all references to Clickhousex

* Document configuration change

* Ensure createdb and migrations can be run in a release

* Remove config added for debug

* Update plausible_variables.sample.env
6 years ago
Uku Taht e5fae1722b Fix trial information being included for subscribed users 6 years ago
Uku Taht 49f153c380
Paginate referrers (#313)
* Add pagination for referrers

* Fix top referrers for goal
6 years ago
Uku Taht 33bf8e8b4d
Add 'This month' and 'Last month' options (#312)
* Add 'This month' and 'Last month'

* Remove 60d option

* Make datpicker transition similar to site switcher
6 years ago
Uku Taht 4aa4dfdcaf
Hash mode (#299)
* Build tracker with hash mode

* Extract hash fragment in hash mode

* Serve new hash-based tracker
6 years ago
Uku Taht d30f17c807
Extract utm_source on the server side (#298)
* Extract utm_source on the server side

* Build tracker scripts
6 years ago
Uku Taht c200b95ce0 Add test for shorthands in external API 6 years ago
Uku Taht 76896f7cee
Show total conversions in addition to uniques (#294) 6 years ago
Uku Taht 01ef17abb0
Make sure all future URLs and sources are decoded (#292) 6 years ago
Uku Taht 43b56d6e61
Clarify subscription (#289)
* Add billing interval to subscription page

* Add test for subscription interval
6 years ago
Uku Taht 248d43ee47
Add page after upgrading to paid plan (#288) 6 years ago
Uku Taht 503dc60b30 Rename Total visitors -> Unique visitors 6 years ago
Uku Taht ff64fccb18
Add site switcher (#281)
* WIP

* Add site switcher

* Run ci

* Highlight active site

* Do not show sites dropdown for logged out users

* Add site switcher to realtime view

* Remove blue border when logged out
6 years ago
Uku Taht 4d11a1d8a1 Fix firewall test 6 years ago
Uku Taht 2e1904e5f0 Add test for firewall 6 years ago
Uku Taht 0a2a2656aa
Page drilldown (#276)
* WIP

* Add filter option for page

* Use JOIN to get correct top numbers

* Filter google search terms by page

* Allow filtering by page from modal

* More robust base url for google search console integration

* Fix tests
6 years ago
Uku Taht f781c5647c
Direct traffic improvements (#268)
* Direct traffic improvements

* Make sure an empty icon shows always

* Enable direct traffic drilldown
6 years ago
Uku Taht 9137fe1fd5
Remove init_admin from test_helper (#267)
* Remove init_admin from test_helper

* Create admin user in test that needs it
6 years ago
Uku Taht e2e901df98
Fix realtime conversion report (#260) 6 years ago
Uku Taht 99bc20a8b3
Referrer drilldown (#252)
* Referrer drilldown

* Render external links for referrer drilldown modal

* Rename top pages to entry pages

* Add google keywords to referrer drilldown

* Fix link in source modal

* Remove back link from referrer drilldown

* Fix tests

* Mix format

* Remove unused function

* Get top stats from the events table

* Revert "Get top stats from the events table"

This reverts commit 67384e2d35.

* Fix pages modal title

* Improve pages report in referrer drilldown

* Fix label in referrer drilldown
6 years ago
Chandra Tungathurthi f7b37fe9ea
Selhosted version Improvements and additional features (#209)
* first commit with test and compile job

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* adding 'prepare' stage

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* updated ci script to include "test" compile phase

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* adding environment variables for connecting to postgresql

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* updated ci config for postgres

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* using non-alpine version of elixir

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* re-using the 'compile' artifacts and added explict env variables for testing

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* removing redundant deps fetching from common code

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* formatting using mix.format -- beware no-code changes!

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* added release config

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* adding consistent env variable for Database

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* more cleaning up of environment variables

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* Adding releases config for enabling releases

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* cleaning up env configs

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* Cleaned up config and prepared config for releases

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* updated CI script with new config for test

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* Added Dockerfile for creating production docker image

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* Adding "docker" build job yay!

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* using non-slim version of debian and installing webpack

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* Adding overlays for migrations on releases

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* restricting the docker built to master branch only

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* typo fix

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* adding "Hosting.md" to explain hosting instructions

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* removed the default comments

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* Added documentation related to env variables

* updated documentation and fixed typo

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* updated documentation

* Bumping up elixir version as `overlays` are only supported in latest version

read release notes: https://github.com/elixir-lang/elixir/releases/tag/v1.10.0

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* Adding tarball assembly during release

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* updated HOSTING.md

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* Added support for db migration

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* minor corrections

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* initializing admin user

Admin user has been added in the "migration" phase. A default user is automatically created in the process. One can provide the related env variables, else a new one will be automatically created for you.

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* Initial base domain update - phase#1

These changes are only meant for correct operating it under self-hosting. There are many other cosmetic changes, that require updates to email, site and other places where the original website and author is used.

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* Using dedicated config variable `base_domain` instead

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* adding base_domain to releases config

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* removing the dedicated config "base_domain", relying on endpoint host

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* Removed the usage of "Mix" in code!

It is bad practice to use "mix" module inside the code as in actual release this module is unavailable. Replacing this with a config environment variable

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* Added support for SMTP via Bamboo Smtp Adapter

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* Capturing SMTP errors via Sentry

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* Minor updates

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* Adding junit formatter -- useful for generating test reports

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* adding documentation for default user

* Resolve "Gitlab Adoption: Add supported services in "Security & Compliance""

* bumping up the debian version to fix issues

fixing some vulnerabilities identified by the scanning tools

* More updates for self-hosting

Changes in most of the places to suit self-hosting. Although, there are some which have been left-off.

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* quick-dirty-fix!

* bumping up the db connect timeout

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* bumping up the db connect timeout

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* bumping up the db connect timeout

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* bumping up timeout - skipping MRs :-/

* removing restrictions on watching for changes

this stuff isn't working

* Update HOSTING.md

* renamed the module name

* reverting formatting-whitespace changes

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* reverting the name to release

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* adding docker-compose.yml and related instructions

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* using `plausible_url` instead of assuming `https`

this is because, it is much to test in local dev machines and in most cases there's already a layer above which is capable for `https` termination and http -> https upgrade

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* WIP: merging changes from upstream

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* wip: more changes

* Pushing in changes from upstream

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* changes to ci for testing

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* cleaning up and finishing clickhouse integration

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* updating readme with hosting details

* removing deleted files from upstream

* minor config adjustments

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* formatting changes

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* changing the connection strategy for clickhouse during release

since clickhouse integration doesn't have an ecto support, we need to prepare the db _before_ the clickhouse migration. One workaround is to connect to a default db on init and then create a db

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* formatting

* cleanup and added separated migration to setup

* Big improvements to selfhosting

- added ability for disabling
  - authentication completely
  - registration
  - landing page

- formatting cleanups

* Big improvements to selfhosting

- added ability for disabling
  - authentication completely
  - registration
  - landing page

- formatting cleanups

* changing smtp auth  to optional

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* removed stale templates and permanently removed landing page

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* removed stale templates and permanently removed landing page

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* removed stale templates and permanently removed landing page

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* WIP

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* fixes form upstream merge

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* added disabling subscription for selfhosted version

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* updated doc

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* formatting

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* Remove reference to file that doesn't exist

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* do not show direct traffic if there's no data

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* addressing PR comments

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* formatting

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>
6 years ago
Uku Taht b254e943bc Add unique visitors to pages modal 6 years ago
Uku Taht 15f6594ddb Add healthcheck endpoint 6 years ago
Uku Taht dc6dfd1694 Add visit duration to referrer drilldown 6 years ago
Uku Taht 7e391fe05e
Visit duration (#227)
* Show average visit duration

* Better format for duration

* Clean up the code for visit duration
6 years ago
Uku Taht 82c7572ba8
Add ability to reset stats (#225)
* Add ability to reset stats

* Fix test
6 years ago
Uku Taht 232298d327
Realtime dashboard (#212)
* Auto-updating dashboard with realtime info

* Remove extra route

* Draw list of countries next to the map

* Nice animations

* Do not show bounce rates in realtime modals

* Update countries and devices in realtime

* Remove unused component

* Show total pageviews in the last 30 minutes

* Show proper labels

* Remove unnecessary z-index

* Fix label for main graph

* Fix compiler warnings

* Add tests

* Fix copy pluralizations

* Fix copy in countries modal

* Real-time -> Realtime

* Looser test assertion

* Show last 30 minutes conversions on realtime report

* Remove EventTarget API because it doesn't work on Safari

* Get referrer drilldown from sessions table

* Fix failing tests
6 years ago
Uku Taht 7683839de1
Fix billing bugs (#208)
* Show past due subscription in user settings

* Users who are past due keep their access

* Add more info in the past due banner

* Show cancelled subscription plan

* Add init admin back into test suite

* Make sure users have access to their stats until the end of billing
cycle

* Fix free subscription bug

* Ensure the latest subscription is loaded
6 years ago
Chandra Tungathurthi 517d4db262
Selfhosted version improvements (#89)
* first commit with test and compile job

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* adding 'prepare' stage

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* updated ci script to include "test" compile phase

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* adding environment variables for connecting to postgresql

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* updated ci config for postgres

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* using non-alpine version of elixir

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* re-using the 'compile' artifacts and added explict env variables for testing

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* removing redundant deps fetching from common code

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* formatting using mix.format -- beware no-code changes!

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* added release config

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* adding consistent env variable for Database

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* more cleaning up of environment variables

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* Adding releases config for enabling releases

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* cleaning up env configs

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* Cleaned up config and prepared config for releases

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* updated CI script with new config for test

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* Added Dockerfile for creating production docker image

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* Adding "docker" build job yay!

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* using non-slim version of debian and installing webpack

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* Adding overlays for migrations on releases

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* restricting the docker built to master branch only

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* typo fix

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* adding "Hosting.md" to explain hosting instructions

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* removed the default comments

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* Added documentation related to env variables

* updated documentation and fixed typo

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* updated documentation

* Bumping up elixir version as `overlays` are only supported in latest version

read release notes: https://github.com/elixir-lang/elixir/releases/tag/v1.10.0

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* Adding tarball assembly during release

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* updated HOSTING.md

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* Added support for db migration

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* minor corrections

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* initializing admin user

Admin user has been added in the "migration" phase. A default user is automatically created in the process. One can provide the related env variables, else a new one will be automatically created for you.

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* Initial base domain update - phase#1

These changes are only meant for correct operating it under self-hosting. There are many other cosmetic changes, that require updates to email, site and other places where the original website and author is used.

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* Using dedicated config variable `base_domain` instead

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* adding base_domain to releases config

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* removing the dedicated config "base_domain", relying on endpoint host

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* Removed the usage of "Mix" in code!

It is bad practice to use "mix" module inside the code as in actual release this module is unavailable. Replacing this with a config environment variable

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* Added support for SMTP via Bamboo Smtp Adapter

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* Capturing SMTP errors via Sentry

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* Minor updates

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* Adding junit formatter -- useful for generating test reports

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* adding documentation for default user

* Resolve "Gitlab Adoption: Add supported services in "Security & Compliance""

* bumping up the debian version to fix issues

fixing some vulnerabilities identified by the scanning tools

* More updates for self-hosting

Changes in most of the places to suit self-hosting. Although, there are some which have been left-off.

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* quick-dirty-fix!

* bumping up the db connect timeout

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* bumping up the db connect timeout

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* bumping up the db connect timeout

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* bumping up timeout - skipping MRs :-/

* removing restrictions on watching for changes

this stuff isn't working

* Update HOSTING.md

* renamed the module name

* reverting formatting-whitespace changes

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* reverting the name to release

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* adding docker-compose.yml and related instructions

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* using `plausible_url` instead of assuming `https`

this is because, it is much to test in local dev machines and in most cases there's already a layer above which is capable for `https` termination and http -> https upgrade

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* WIP: merging changes from upstream

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* wip: more changes

* Pushing in changes from upstream

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* changes to ci for testing

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* cleaning up and finishing clickhouse integration

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* updating readme with hosting details

* removing deleted files from upstream

* minor config adjustments

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* formatting changes

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* changing the connection strategy for clickhouse during release

since clickhouse integration doesn't have an ecto support, we need to prepare the db _before_ the clickhouse migration. One workaround is to connect to a default db on init and then create a db

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* formatting

* cleanup and added separated migration to setup

* Big improvements to selfhosting

- added ability for disabling
  - authentication completely
  - registration
  - landing page

- formatting cleanups

* Big improvements to selfhosting

- added ability for disabling
  - authentication completely
  - registration
  - landing page

- formatting cleanups

* changing smtp auth  to optional

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* removed stale templates and permanently removed landing page

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* removed stale templates and permanently removed landing page

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>
6 years ago
Uku Taht 0e17a4ecfd Remove google auth from a site 6 years ago
Uku Taht 9a91c6d901 Allow custom domain to be deleted 6 years ago
Uku Taht 80b98762f4
More robust scheduling for email reports (#189)
* More robust scheduling for email reports

* Update config for new email report jobs

* Only build docker on master

* Use Postgres 12 in travis

* Cache elixir files

* use Postgres 12 in travis

* Specify env on one line

* Wrong postgres version
6 years ago
Uku Taht 77e8276673 Ignore malformed referrer URL 6 years ago
Uku Taht c5e4fb50d5 Include new plans in trial upgrade email 6 years ago
Uku Taht 6af6538a01 Fix tests 7 years ago
Uku Taht b9231b0cd0 Fix email reports 7 years ago
Uku Taht a12cfa8a8f
Maxmind (#80)
* Add maxmind Geolite2 database

* Look up the actual country from MaxMind

* Fix test for country lookup
7 years ago
Uku Taht 1c9ad7e778 Remove initial referrer from Clickhouse 7 years ago
Chandra Tungathurthi b4b7532f07
Formatting only changes - No code change (#75)
* first commit with test and compile job

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* adding 'prepare' stage

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* updated ci script to include "test" compile phase

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* adding environment variables for connecting to postgresql

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* updated ci config for postgres

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* using non-alpine version of elixir

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* re-using the 'compile' artifacts and added explict env variables for testing

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* removing redundant deps fetching from common code

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* formatting using mix.format -- beware no-code changes!

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* added release config

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* adding consistent env variable for Database

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* more cleaning up of environment variables

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* Adding releases config for enabling releases

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* cleaning up env configs

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* Cleaned up config and prepared config for releases

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* updated CI script with new config for test

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* Added Dockerfile for creating production docker image

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* Adding "docker" build job yay!

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* using non-slim version of debian and installing webpack

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* Adding overlays for migrations on releases

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* restricting the docker built to master branch only

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* typo fix

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* adding "Hosting.md" to explain hosting instructions

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* removed the default comments

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* Added documentation related to env variables

* updated documentation and fixed typo

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* updated documentation

* Bumping up elixir version as `overlays` are only supported in latest version

read release notes: https://github.com/elixir-lang/elixir/releases/tag/v1.10.0

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* Adding tarball assembly during release

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* updated HOSTING.md

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* Added support for db migration

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* minor corrections

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* initializing admin user

Admin user has been added in the "migration" phase. A default user is automatically created in the process. One can provide the related env variables, else a new one will be automatically created for you.

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* Initial base domain update - phase#1

These changes are only meant for correct operating it under self-hosting. There are many other cosmetic changes, that require updates to email, site and other places where the original website and author is used.

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* Using dedicated config variable `base_domain` instead

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* adding base_domain to releases config

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* removing the dedicated config "base_domain", relying on endpoint host

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* Removed the usage of "Mix" in code!

It is bad practice to use "mix" module inside the code as in actual release this module is unavailable. Replacing this with a config environment variable

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* Added support for SMTP via Bamboo Smtp Adapter

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* Capturing SMTP errors via Sentry

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* Minor updates

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* Adding junit formatter -- useful for generating test reports

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* adding documentation for default user

* Resolve "Gitlab Adoption: Add supported services in "Security & Compliance""

* bumping up the debian version to fix issues

fixing some vulnerabilities identified by the scanning tools

* More updates for self-hosting

Changes in most of the places to suit self-hosting. Although, there are some which have been left-off.

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* quick-dirty-fix!

* bumping up the db connect timeout

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* bumping up the db connect timeout

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* bumping up the db connect timeout

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* bumping up timeout - skipping MRs :-/

* removing restrictions on watching for changes

this stuff isn't working

* Update HOSTING.md

* renamed the module name

* reverting formatting-whitespace changes

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* reverting the name to release

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* adding docker-compose.yml and related instructions

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* using `plausible_url` instead of assuming `https`

this is because, it is much to test in local dev machines and in most cases there's already a layer above which is capable for `https` termination and http -> https upgrade

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* WIP: merging changes from upstream

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* wip: more changes

* Pushing in changes from upstream

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* changes to ci for testing

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* cleaning up and finishing clickhouse integration

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* updating readme with hosting details

* removing deleted files from upstream

* minor config adjustments

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* formatting changes

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>
7 years ago
Uku Taht 933cff6fe0
Remove stats from postgres (#74)
* Remove stats code related to postgres events and sessions

* Use Clickhouse events in fetch tweets worker
7 years ago
Uku Taht 1efc68fa59 Fix ssh test 7 years ago
Uku Taht 34d7d12229 Use SSH in Elixir for custom domains 7 years ago
Uku Taht 761072d285 Fix bug with `has_pageviews?` 7 years ago
Uku Taht 25d8f2f3f9
Schedule regular jobs with Oban (#69)
* Use oban for work

* Remove mix task for email reports

* Convert remaining tasks to Oban workers

* Add crontab for scheduled jobs

* Only enable cron if configured to do so
7 years ago
Chandra Tungathurthi 5eb8929929
Support for docker based self-hosting (#64)
* first commit with test and compile job

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* adding 'prepare' stage

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* updated ci script to include "test" compile phase

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* adding environment variables for connecting to postgresql

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* updated ci config for postgres

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* using non-alpine version of elixir

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* re-using the 'compile' artifacts and added explict env variables for testing

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* removing redundant deps fetching from common code

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* formatting using mix.format -- beware no-code changes!

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* added release config

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* adding consistent env variable for Database

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* more cleaning up of environment variables

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* Adding releases config for enabling releases

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* cleaning up env configs

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* Cleaned up config and prepared config for releases

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* updated CI script with new config for test

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* Added Dockerfile for creating production docker image

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* Adding "docker" build job yay!

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* using non-slim version of debian and installing webpack

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* Adding overlays for migrations on releases

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* restricting the docker built to master branch only

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* typo fix

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* adding "Hosting.md" to explain hosting instructions

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* removed the default comments

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* Added documentation related to env variables

* updated documentation and fixed typo

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* updated documentation

* Bumping up elixir version as `overlays` are only supported in latest version

read release notes: https://github.com/elixir-lang/elixir/releases/tag/v1.10.0

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* Adding tarball assembly during release

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* updated HOSTING.md

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* Added support for db migration

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* minor corrections

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* initializing admin user

Admin user has been added in the "migration" phase. A default user is automatically created in the process. One can provide the related env variables, else a new one will be automatically created for you.

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* Initial base domain update - phase#1

These changes are only meant for correct operating it under self-hosting. There are many other cosmetic changes, that require updates to email, site and other places where the original website and author is used.

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* Using dedicated config variable `base_domain` instead

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* adding base_domain to releases config

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* removing the dedicated config "base_domain", relying on endpoint host

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* Removed the usage of "Mix" in code!

It is bad practice to use "mix" module inside the code as in actual release this module is unavailable. Replacing this with a config environment variable

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* Added support for SMTP via Bamboo Smtp Adapter

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* Capturing SMTP errors via Sentry

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* Minor updates

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* Adding junit formatter -- useful for generating test reports

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* adding documentation for default user

* Resolve "Gitlab Adoption: Add supported services in "Security & Compliance""

* bumping up the debian version to fix issues

fixing some vulnerabilities identified by the scanning tools

* More updates for self-hosting

Changes in most of the places to suit self-hosting. Although, there are some which have been left-off.

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* quick-dirty-fix!

* bumping up the db connect timeout

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* bumping up the db connect timeout

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* bumping up the db connect timeout

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* bumping up timeout - skipping MRs :-/

* removing restrictions on watching for changes

this stuff isn't working

* Update HOSTING.md

* renamed the module name

* reverting formatting-whitespace changes

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* reverting the name to release

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* adding docker-compose.yml and related instructions

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* using `plausible_url` instead of assuming `https`

this is because, it is much to test in local dev machines and in most cases there's already a layer above which is capable for `https` termination and http -> https upgrade

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* WIP: merging changes from upstream

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* wip: more changes

* Pushing in changes from upstream

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* changes to ci for testing

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* cleaning up and finishing clickhouse integration

Signed-off-by: Chandra Tungathurthi <tckb@tgrthi.me>

* updating readme with hosting details
7 years ago
Uku Taht 43b46e7607 Show last touch referrer for drilldown 7 years ago
Uku Taht a228c6162f Default to empty pathname 7 years ago
Uku Taht b31c0114c5 Use Clickhouse everywhere 7 years ago
Uku Taht d94a8dbc7e
Clickhouse (#66)
* Get stats from clickhosue

* Pull stats from clickhouse

* Use correct Query namespace

* Use Clickhouse in unit tests

* Use Clickhouse in stats controller tests

* Use fixtures for unit tests

* Add Clickhouse to travis

* Use Clickhouse session store for sessions

* Add garbage collection to session store

* Reload session state from Clickhouse on server restart

* Query from sessions table

* Trap exits in event write buffer

* Run hydration without starting the whole app

* Make session length 30 minutes

* Revert changes to fingerprint schema

* Remove clickhouse from fingerprint sessions

* Flush buffers before shutdown

* Use old stats when merging

* Remove old session schema

* Fix tests with CH sessions

* Add has_pageviews? to Stats

* Use CH in staging

* Update schema

* Fix test setup
7 years ago
Uku Taht d1b703f007
Clickhouse (#63)
* Get stats from clickhosue

* Pull stats from clickhouse

* Use correct Query namespace

* Use Clickhouse in unit tests

* Use Clickhouse in stats controller tests

* Use fixtures for unit tests

* Add Clickhouse to travis

* Use Clickhouse session store for sessions

* Add garbage collection to session store

* Reload session state from Clickhouse on server restart

* Query from sessions table

* Trap exits in event write buffer

* Run hydration without starting the whole app

* Make session length 30 minutes

* Revert changes to fingerprint schema

* Remove clickhouse from fingerprint sessions

* Flush buffers before shutdown

* Use old stats when merging

* Remove old session schema

* Fix tests with CH sessions

* Add has_pageviews? to Stats
7 years ago
Uku Taht 5d1dc0d27c
Clickhouse DB (#62)
* WIP

* Get all stats from Clickhouse

* Use https dependency

* Do not namespace db tables in hydrate task

* Update hydration task

* Ingest data to clickhouse

* Double-write to both Clickhouse and Postgres

* Add test setup

* Keep old stats module

* Prepare for live ingestion test
7 years ago
Uku Taht 9476ccf95c
Yearly billing (#58)
* Add yearly billing options

* Add new subscription names

* Add note about yearly billing in the email

* Add a test for change_plan function

* Add allowance function back in

* Test billing controller endpoint
7 years ago
Uku Taht 4163d2a9ba
Authentication updates (#54)
* Use separate cookie for shared link auth

* Remove CSRF protection from login and register pages

* Add AuthorizeStats Plug

* Fix tests
7 years ago
Uku Taht ddf08b97e8 Cascade custom domain deletion 7 years ago
Uku Taht 2b50c6cf2a Use correct headers 7 years ago
Uku Taht 074e895204 redirect users to /sites after login 7 years ago
Uku Taht 2cf57a795f Drop session 7 years ago
Uku Taht e14001664c Fix tests 7 years ago
Uku Taht 594b085467
Analytics without using cookies (#44)
* Use fingerprints instead of user_id

* Fix tests
7 years ago
Uku Taht 6d1f7afde5 Do not send check stats email to someone who has weekly report enabled 7 years ago
Uku Taht ee92ed5213
Email flows (#46)
* Update activation email

* Update trial upgrade emails

* Remove feedback emails

* Remove feedback email test

* Update welcome email (and send it earlier)

* Site setup emails

* Add create site emails for trialers

* Add check stats email

* Remove dead code

* Adjust site setup emails for users who are not on trial

* BCC myself on all new emails

* Use old phrasing in the welcome email
7 years ago
Uku Taht 3fc17a51d5
Add field to track trial expiry date (#45) 7 years ago
Uku Taht a2ed7514d7
Improve datepicker (#41)
* Update datepicker dropdown style

* Add custom range for datepicker
7 years ago
Uku Taht cc5722050a
Show favicons for referrers (#39) 7 years ago
Uku Taht c0f686b3d0 Fix trailing slash in referrer 7 years ago
Uku Taht 08eba461df Fix issue with snippet code 7 years ago
Uku Taht 79b9f72b52
Custom domains (#34)
* UI to create custom domains

* Only call ssh once per domain

* Update copy for custom domain setup

* Use correct user for ssh
7 years ago
Uku Taht 5495e53454
Backfill fingerprints for returning users (#33) 7 years ago
Uku Taht ed3110a6ea
Introduce initial referrer and initial referrer source (#32)
* Introduce initial referrer and initial referrer source

* Add test
7 years ago
Uku Taht 8a4d0a91d1
WIP: New UI updates (#19)
* WIP: New UI idea

* Remove hover styling if things are not clickable

* Improve mobile view

* Remove dead code

* Fade in tabs

* Update landing page with new style

* Fix countries test

* Fix alignment in conversions report
7 years ago
Uku Taht 3679a7351f Remove www from domain 7 years ago
Uku Taht 7dbbc8ba22
Configurable site id (#30)
* Use site id instead of hostname for events

* Use site id in domain status check

* Revert change to tracking module

* Catch more places where link generation needed updating

* Rename site_id to domain

* Drop hostname index from events
7 years ago
Uku Taht ad2e5b6b28 Remove unused code 7 years ago
Uku Taht f9724bf9f1 Delete user subscription when deleting the user 7 years ago
Uku Taht 9b0bd2162c Remove unused socket code 7 years ago
Uku Taht 614e2eeb34 Remove unused helper function 7 years ago
Uku Taht 5f62025176
Shared links (#29)
* Create shared links UI

* Show shared links in a list on settings page

* Show dropdown for each shared link

* Show icon actions for shared links

* Log user in when they click non-password-protected shared link

* Actually authenticate using a password

* Delete shared links
7 years ago
Uku Taht c96f364ad8
Add and remove recipients for email reports (#28)
* Add and remove recipients for email reports

* Remove unused google_settings controller action

* Background job sends email reports to multiple recipients

* Add a way to unsubscribe for recipients who cannot log in

* Fix view on plausible link

* Include bounce rate in email report
7 years ago
Uku Taht 423db302f7 Only count new visitors in referrer report 7 years ago
Uku Taht 9df398b663 Use group by for grouping tweets 7 years ago
Uku Taht b02cb74181
Fetch and display tweets (#27) 7 years ago
Uku Taht 94a20fb0a2
Add exit page to sessions (#25) 7 years ago
Uku Taht 9f30e9eb1c
Export main graph as csv (#24)
* Export main graph as csv

* Remove unused deps from mix-lock

* Update bamboo dependency
7 years ago
Uku Taht 0f6d742ae1 Show 6 months instead of 3 7 years ago
Uku Taht 5631367aef Add 'last year' instead of '6 months' 7 years ago
Uku Taht 254d34d0a4 Trim trailing slash from referrer 7 years ago
Uku Taht 29ae3a2c21 Show bounce rate for referrers and pages 7 years ago
Uku Taht df1ba2cd82 Remove session length from the UI 7 years ago
Uku Taht 980275092d Show bounce rate and session length 7 years ago
Uku Taht 438a1c3d19 Cascade google auth deletion 7 years ago
Uku Taht b7815bfe25 Revert "Cascade site google auth deletion"
This reverts commit fd8917d4f3.
7 years ago
Uku Taht fd8917d4f3 Cascade site google auth deletion 7 years ago
Uku Taht 3548c57710 Capture session start 7 years ago
Uku Taht f7d752988a Add sessions 7 years ago
Uku Taht 8888c86820 Launch goals & conversions 7 years ago
Uku Taht e0737849ba Add test for conversion in month 7 years ago
Uku Taht 5ee2d908f2 Show conversion rate in the top stats when filtered for goal 7 years ago
Uku Taht c540c70e87 Show conversion rate when filtering for goal 7 years ago
Uku Taht b5797c8b10 Test getting conversions with a goal filter 7 years ago
Uku Taht f71f98a521 Test Google referrer drilldown 7 years ago
Uku Taht 9feecf6741 Test pageview goal filter for referrers 7 years ago
Uku Taht 0ada544fd6 Test query for 6 months 7 years ago
Uku Taht 5d820dea39 Add test for goal filter 7 years ago
Uku Taht 96abfc4ea1 Click on goal to see conversions for it 7 years ago
Uku Taht e8f20e67cc
React (#17)
* Load dashboard with react

* Rename stast2 to dashboard

* Save timeframe on the frontend

* Implement current visitors

* Implement comparisons

* React to route changes

* Add modals

* Show number of visitors on hover

* Show 'Today' for today

* Add 30 days

* Show referrer drilldown

* Arrow keys to go back and forward

* Improve comparisons UI

* Fix dropdown when clicking on it

* Verify API access in a memoized fashion

* Test API access

* Test stats through controller

* Move map formatting from stats controller to stats

* Remove unused code

* Remove dead code from query

* Remove dead code from stats templates

* Add stats view test back in

* Render modal inside the modal component

* Implement google search terms

* Add explanation for screen sizes

* Separate dashboard JS from the app js
7 years ago
Uku Taht 0ae5448a2b Show percentages for most stats 7 years ago
Uku Taht 9f2cdea503 Fix timezone display 7 years ago
Uku Taht 0b96ed3b6c Remove goal name 7 years ago
Uku Taht 71f6ccc73c Add tests for site controller 7 years ago
Uku Taht bf8443d028 Test goal_conversions function 7 years ago
Uku Taht c7ed0d54ce Add goals and conversions 7 years ago
Uku Taht d687663b80 Use new /event endpoint for pageviews 7 years ago
Uku Taht 4adfa6bdfa Add tests for untested parts of stats.ex 7 years ago
Uku Taht 9dd08ddfa6 Add test for change plan page 7 years ago
Uku Taht 3ea64de42b Add `name` to events 7 years ago
Uku Taht 0c453be08c Rename pageviews to events 7 years ago
Uku Taht 7c7bdc6ef6 Proper timestamp 7 years ago
Uku Taht 71edf2c86d Remove unused fields from pageviews 7 years ago
Uku Taht b458b5a392 Fix webhook 7 years ago
Uku Taht 1bb756362d Use email if user id is not known 7 years ago
Uku Taht 00d02faad0 Show current visitors 7 years ago
Uku Taht d2855210de Fix bug with screen size 7 years ago
Uku Taht 779d64e19a Initial commit 7 years ago