tools

VIM: Better grepping (in Windows)

Several times I noticed that grepping in VIM (finding in files) does not work very well comparing to Visual Studio or VSCode at least. I felt it pretty hard last time when I tried to search in a medium-size web project. It took over 23 seconds to search for files and another 55 seconds to render the list of results. I’ve never experienced something similar in any other editor.  Another think that bothered me was the quality of results which frequently showed binary files, multi-line records or everything else what I could call “rubbish” in the context of file search option:

2017-04-08_22-41-56

The first guess is that alternative editors just skip files that are not source code. And that’s the thing (at least looking at Visual Studio Code file search):

2017-04-08_22-54-55

Unfortunately there is no easy way to tell VIM grepper to exclude locations from search (at least in Windows). The default findstr (you can find more about it in my previous post) does not have such option. Nevertheless there is a way to achieve such behaviour. Just add this snippet to your $VIMRC file:

function! FastGrep(what, ...) abort
 let where = a:0
 if where == ""
    let where = "*.*"
 endif
 let command = "silent ! dir /s/b/a:-d"
 let command = command . " | findstr /v \\node_modules\\"
 let command = command . " | findstr /v \\dist\\"
 let command = command . " | findstr /v \\\.git\\"
 let command = command . " | findstr /v \.map$"
 let command = command . " | findstr /v \.swp$"
 let command = command . " | findstr /v \.ts\.html$"
 let command = command . " | findstr /v ~$"
 " add excludes here ------^
 let command = command . " > \\%home\\%\\solution_files.tmp"
 execute comma              nd
 execute "grep /f:\\%home\\%\\solution_files.tmp ".a:what." ".where
endfunction

:command! -nargs=+ -complete=command Grep2 call FastGrep(<q-args>)

The idea behind the snippet is the following: get list of all the files in the working folder, remove the locations you want to exclude, and then pass the list of files to “findstr” command (/f switch allows to pass the list of input files).

Here’s a quick list of tricks used in the snippet:

execute              - run vim command
!                    - execute shell command
silent               - skip "Press ENTER to continue" after running shell
dir /s/b/a:-d        - get a raw list (/b) of files only (/a:-d) 
                       in all subdirectories (/s) 
findstr /v \.git\    - find lines that does not containing "\.git\"
%home%               - user home directory (i.e. c:\users\user_1)
FastGrep(what, ...)  - function taking one named argument and array of 
                       unnamed arguments
a:0                  - first unnamed argument
:command!            - define VIM command alias
-nargs=+             - alias must have arguments (any number)
<q-args>             - list of arguments passed to the alias

Now in order to execute such command you just need to run:

:Grep2 myFunction1
:Grep2 myFunction2 *.js

The same operation that took initially 23 seconds was executed in less than 1 second (with rendering results). Of course it showed less records, but only the meaningful ones (a side note: my medium-size web application contained 996 folders inside /node_modules !!).

Remember that you can extend it by your own excludes.

tools, Uncategorized

VIM: Find in files

It’s not a surprise that there is a built-in way to find in files in VIM. You can use four commands: :grep, :lgrep, :vimgrep, :lvimgrep.

2017-04-02_09-19-29

:grep will use the default “find” tool for your operating system (“grep” for linux family, “findstr” for Windows). :vimgrep will use built-in vim search (same as for “/” searching) which is slower than system search (but works if you have problems with grep/findstr/other). On the other hand vimgrep has the same syntax everywhere (:grep depends on external tool used). Search result is presented using Quickfix window (opened with :copen). :lgrep/:lvimgrep will do the same, but using location list (opened with :lopen). In practise the difference is that if you use Quickfix window (:copen) it will stick to the current editor. Location window will show in the same way, but if you select anything then the window will disappear and you will have to run :lopen again.

I guess you have to choose yourself which command you like most.  The syntax is as follows:

:grep searchpattern locationpattern

i.e.:

:grep MyMethod *.*

Will look for phrase “MyMethod” in all files in current workind directory (you can check currect directory using :pwd). By default it does not perform recursive search (it will not search in subdirectories). And here the fun part starts.

In order to search in subdirectories you would have to use **/*.* pattern. However this works for :vimgrep and :grep in linux, but does not work for Windows. In order to make it work on Windows you would need to use /S parameter (linux grep also accepts -R parameter which does the same thing). You can pass it like that:

:grep /S MyMethod *.*

This will work on Windows and will find all “MyMethod” phrases in all files in all subdirectories of current working directory. However you can still make it the default behaviour in VIM. But first you neeed to know one thing:

:set grepprg

This is a variable that controls the usage of external grep tool in VIM. Depending on your operating system you will have different results:

grepprg=findstr /n    # <--- on Windows
grepprg=grep -n $* /dev/null # <---- on Linux

So in order to use recursive search you have to modify grepprg and add a proper parameter:

grepprg=findstr /S /n                  # <--- on Windows
grepprg=grep -nR $* /dev/null          # <---- on Linux

The same way you can add more parameters or make more complex commands.

tools

CMDer/ConEmu – starting new tab in current folder

Recently I posted about a trick allowing the user to open new CMDer/ConEmu tab in current folder from command line. In response ConEmu team twitted a better solution that I would like to share:

2017-03-30_22-00-53

I don’t want to have specific shortcut for each task so I just edited default {cmd} task this way:

2017-03-30_21-51-35

The key here is to add “/dir “%CD%” as task parameter and remove “:d:”%USERPROFILE%” parameter from the textbox on the bottom. Just after that when you add new tab (in my case Ctrl + Shift + T) you can specify “Startup directory for new process” or leave it empty. In the latter case (and only then!) ConEmu will open new task in current directory.

2017-03-30_22-04-17

Why is this better?

The previous solution required active console in order to execute some command. Now everything is done by ConEmu, so you can have some process running (web server/test watch/vim etc) and at the same time start new tab in the same folder.

Thanks a lot ConEmu Team!

tools, Uncategorized

CMDer/ConEmu – starting new tab in current folder from command line

Quite frequently I found myself opening several console tabs in the same folder (for vim, for running unit tests, for git, for executing the application etc). Starting new tab and navigating to my project’s directory is not optimal so I created a short !cmd.bat snippet that I can run everywhere (added as global alias) and it will create a new tab for me in the same directory. Here’s the content:

cmd /k "%ConEmuDir%\..\init.bat" -new_console:C:"%CMDER_ROOT%\cmder.exe"

It’s taken from “ConEmu -> Settings -> Tasks” with slight modification (adding tab icon). Here’s explanation of switches used by the snippet:2017-03-26_23-52-18

/k - run initial script just after running console
-new_console - open process in a new tab
:C:"file location" - path to file containing tab icon

Alternative way is to use shortcut for creating new tab and then selecting current folder from drop-down menu (see below).

2017-03-26_23-53-00

Unfortunately because of some bug it does not work in my current version of Cmder/ConEmu (build 161022). The other this is that it requires more clicking 🙂

Uncategorized

Review/Planning no. 0

Review

The following features have been developed so far:

  • The app reads the structure of given folder
  • It saves the structure to a file
  • It calculates hash of every file
  • Other: Environment configuration (VIM + dotnet core SDK 1.0.1)

Some of the features delivered here does not bring real value i.e. calculating file hash or saving data to a file. This highlights the lack of planning last time. Up till now the app is not useful and does not solve any real problem.

Plannig

The goal of the next iteration is to create functionality to move media files from source folder to target, applying some simple file reorganisation.
Files are organised by file’s creation year and month. Files structure from source location should be preserved. In order to avoid conflicts/errors the target directory must be empty.
The operation must generate some kind of a journal so that the operation can be rolled back or resumed if needed.

Tasks:

  • Generate dry-run file in order to review the moving/organisation process.
    File moving cannot be done on-the-fly. This operation can take a while and it may mess up files structure, so it’s better first to review the file showing suggested list of “source location -> target location” entries.
  • Generate journal file based on source file structure.
    The application should always leave track of every change as it is about moving sensitive data (personal media files). In case of any issues it should be possible to resume the operation or roll it back.
  • Move files to proper folders updating journal.
    Moving physical files from one place to another, based on generated dry-run file.

Currently I do not set the date of next Review/Planning. I expect to have it at latest in 3 weeks but I want to check my velocity first.

Uncategorized

DSP2017 – What’s your methodology?

Almost three weeks have passed since the start of DSP2017. I cannot say that I feel excited about what I did. Nothing really changed since the last year (which was unsuccessful). Maybe it’s time for a quick recap? Maybe the way I drive my project after-hours is less serious than then one I do at work? Why? Wiseman said: Pay yourself first.

Maybe the thing is that I miss some kind of a process, rules to proceed, methodology? Why not to take the best of what I do and practice at work?

 

Talking about me, from now on I want to introduce some scrum ceremonies into my project. I do want to have iterations. And what’s related I want to have a review session followed by planning for the next iteration. I want to make it public, so expect Review/Planning post every iteration (I guess it will be done bi-weekly but this may change).

 

I feel I need review session to motivate myself. And nothings is a better motivation than making the progress public. Praise if it’s good, sling mud at me if it’s bad. Let it be colorful with clear decision that I want to go on or not.

 

Planing is also crucial for such project. It makes me know what to do next, so I can focus on coding instead of looking for the next thing to do. The worst excuse for not planning is not having time for it. I was about wasting a lot of time for doing stuff which turned out to be low-priority. Not only once.

Expect the first post soon.

tools

Using several dotnet CLI versions and why you should care

Frankly speaking “several” keyword in the title is an overuse, because while writing this article I’m thinking only about two versions of dotnet CLI. Why? Because RC4 version dropped support to project.json and switched completely to the new *.csproj format. On one hand it’s good – let’s follow the hype, cool to have tooling already. On the other hand if you still maintain/develop solution based on project.json then you have a problem (you will not be able to use dotnet CLI with that).

I am in such case and have found a solution that might be worth sharing.

Prerequisites

Download dotnet CLI binaries (without installer) and extract package to some known folder. In my case it was:

C:\tools\dotnet-prev2
C:\tools\dotnet-rc4

Overriding PATH variable

The trick is to switch PATH variable to proper dotnet CLI location. Of course you could copy paste specific “set PATH=…” expression but some script may come in handy. Below is the script that works for me (I use an alias “set-dotnet.cmd”).  Remember to add script location to PATH (you may want to see this post).

@echo off
set _DOTNET2=c:\Tools\dotnet-prev2;
set _DOTNET4=c:\Tools\dotnet-rc4;
:: Batch way of string replace
set path=%path:C:\Tools\dotnet-prev2;=%
set path=%path:C:\Tools\dotnet-rc4;=%

if [%1]==[] goto usage
if "%1"=="prev2" goto prev2
if "%1"=="rc4" goto rc4
goto unknown

:prev2
@echo you selected prev2
set path=%_DOTNET2%%PATH%
exit /B 0

:rc4
@echo you selected rc4
set path=%_DOTNET4%%PATH%
exit /B 0

:unknown
@echo Unknown parameter %1
goto :usage

:usage
@echo Usage: %0 ^<ver^>
@echo        where ^<ver^> is one of: prev2, rc4
exit /B 1

This lets me switch PATH to specific dotnet version using:

set-dotnet prev2
set-dotnet rc4

Normally I do scripting in Powershell but things related to environment variables are done easier in batch scripting – neither setx nor [Environment]::SetEvironmentVariable() affect currently active console (challenge me in comments if you want).

As batch scripting is pretty exotic some explanation is needed:

Exit /b 0

This means exiting current subrouting which is the whole batch script in case of “goto” or returning from “call” (0 is errorlevel)

:: Batch way of string replace
%variable:findtext=replacewith%

This is a way to do string replace withing variable. Let’s have another example

set x=hello world
set x=%x:world=chriss%

Executing these two lines above results in x=hello chriss

Cheers!

PS: Refactoring

Further refactoring can lead to having much simpler version which is having two batch scripts:

set-dotnet-prev2.cmd:

set path=c:\Tools\dotnet-prev2;%PATH%

set-dotnet-rc4.cmd:

set path=c:\Tools\dotnet-rc4;%PATH%

As an extra credit this will enable autocomplete for set-dotnet-*